connpool

package module
v0.0.0-...-145e103 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 1, 2025 License: MIT Imports: 7 Imported by: 0

README

Connection Pool (connpool)

A generic, thread-safe connection pool implementation for managing and reusing connections with features like health checks, size limits, idle connection management, and metrics tracking.

Features

  • Thread-safe: Designed for concurrent use across multiple goroutines.
  • Connection Lifecycle Management: Automatic creation (Factory), reuse (Put), and closing (Close).
  • Configurable Limits:
    • MaxSize: Maximum total connections (active + idle).
    • MaxIdle: Maximum idle connections to keep.
    • MaxIdleTime: Maximum time a connection can remain idle before being closed by cleanup.
    • MaxAge: Maximum time a connection can exist before being closed by cleanup.
  • Connection Health Checking:
    • Optional Ping function to verify connection health.
    • Automatic periodic health checks for idle connections (HealthCheckInterval, HealthCheckTimeout).
    • Optional ping-on-get (performed when Ping is set).
  • Idle Connection Cleanup: Background goroutine removes connections exceeding MaxIdleTime or MaxAge.
  • Waiting Mechanism: Get blocks and waits if the pool is at MaxSize until a connection is returned or the context times out.
  • Context Awareness: Get, Close, Factory, and Ping operations respect context.Context cancellation and deadlines.
  • Explicit Discard: Discard method to remove known broken connections from the pool.
  • Comprehensive Metrics: Tracks gets, puts, hits, misses, waits, timeouts, connection closures, and more via Stats().
  • Graceful Shutdown: Close method waits for active connections to be returned (respecting context).

Installation

go get github.com/briceamen/connpool

Usage

package main

import (
	"context"
	"net"
	"time"
	
	"github.com/briceamen/connpool"
)

func main() {
	// Define connection factory
	factory := func(ctx context.Context) (interface{}, error) {
		return net.Dial("tcp", "example.com:80")
	}
	
	// Define connection close function
	closeFunc := func(conn interface{}) error {
		return conn.(net.Conn).Close()
	}
	
	// Optional ping function to check connection health
	pingFunc := func(conn interface{}) error {
		return nil // Implement real connection check
	}
	
	// Create pool configuration
	config := connpool.Config{
		Factory:      factory,
		Close:        closeFunc,
		Ping:         pingFunc,
		MaxSize:      10,
		MaxIdle:      5,
		MaxIdleTime:  time.Minute * 5,
	}
	
	// Initialize the pool
	pool, err := connpool.NewPool(config)
	if err != nil {
		panic(err)
	}
	defer pool.Close(context.Background())
	
	// Get a connection
	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	
	conn, err := pool.Get(ctx)
	if err != nil {
		panic(err)
	}
	
	// Use the connection...
	
	// Return the connection to the pool when done
	pool.Put(conn)
	
	// Or discard it if there was an error
	// pool.Discard(conn)
	
	// Check metrics
	stats := pool.Stats()
	println("Total connections:", pool.Len())
	println("Idle connections:", pool.IdleLen())
	println("Hits:", stats.Hits)
	println("Misses:", stats.Misses)
}

Configuration

The Config struct allows you to configure the pool behavior:

Field Description
Factory Function that creates new connections (required)
Close Function that closes connections (required)
Ping Optional function to check connection health
MaxSize Maximum number of connections in the pool (active + idle)
MaxIdle Maximum number of idle connections to keep
MaxIdleTime Maximum time a connection can remain idle before being closed by the background cleanup goroutine
MaxAge Maximum time a connection can exist before being closed by the background cleanup goroutine
HealthCheckInterval Interval for performing background health checks (Ping) on idle connections
HealthCheckTimeout Timeout duration for a single background health check (Ping)
CleanupInterval Explicit interval for the background cleanup goroutine (overrides defaults based on MaxIdleTime/HealthCheckInterval)

Error Handling

The package defines several errors in errors.go:

  • ErrPoolClosed: Operation attempted on a closed pool
  • ErrFactoryRequired: Pool initialized with nil Factory
  • ErrCloseFuncRequired: Pool initialized with nil CloseFunc
  • ErrInvalidConnection: Invalid connection passed to Put/Discard
  • ErrInvalidConfigMaxSize: MaxSize is set to 0
  • ErrGetTimeout: Get timed out waiting for a connection
  • ErrCloseTimeout: Close timed out waiting for connections

Metrics

The Metrics struct provides detailed statistics about pool operations:

  • Gets: Total number of Get calls
  • Puts: Total number of Put calls
  • Discards: Total number of Discard calls
  • Hits: Number of times Get returned an idle connection
  • Misses: Number of times Get created a new connection
  • Waits: Number of times Get had to wait
  • Timeouts: Number of times Get timed out
  • PingFailures: Number of connections discarded due to failed ping
  • IdleClosed: Number of idle connections closed by the background cleanup (MaxIdleTime)
  • AgeClosed: Number of connections closed due to MaxAge
  • FactoryFailures: Number of times the factory failed to create a connection
  • CloseTimeouts: Number of times Close timed out waiting for active connections
  • MaxActive: Maximum active connections observed

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrPoolClosed is returned when an operation is attempted on a closed pool.
	ErrPoolClosed = errors.New("connpool: pool closed")

	// ErrFactoryRequired is returned when initializing a pool with a nil Factory.
	ErrFactoryRequired = errors.New("connpool: factory function is required")

	// ErrCloseFuncRequired is returned when initializing a pool with a nil CloseFunc.
	ErrCloseFuncRequired = errors.New("connpool: close function is required")

	// ErrInvalidConnection is returned by Put or Discard if the provided connection
	// is nil, does not belong to this pool, or is already closed.
	ErrInvalidConnection = errors.New("connpool: invalid connection")

	// ErrGetTimeout is returned by Get when the context times out waiting for a connection.
	// Note: Currently, Get wraps context.DeadlineExceeded or context.Canceled instead.
	ErrGetTimeout = errors.New("connpool: get timed out waiting for connection")
)

Functions

This section is empty.

Types

type CloseFunc

type CloseFunc func(conn interface{}) error

CloseFunc defines a function that closes a connection. This function should handle the necessary cleanup for the connection.

type Config

type Config struct {
	// Factory is the function used to create new connections.
	// This field is required.
	Factory Factory

	// Close is the function used to close connections.
	// This field is required.
	Close CloseFunc

	// Ping is an optional function used to check the health of a connection.
	// It is called before returning a connection from Get (if the connection is not new)
	// and periodically on idle connections if HealthCheckInterval is set.
	// If Ping returns an error, the connection is discarded.
	Ping PingFunc

	// MaxSize is the maximum number of connections the pool can manage.
	// A value of 0 or less means no limit (unlimited connections).
	// This includes both idle and active connections.
	MaxSize int

	// MaxIdle is the maximum number of idle connections to keep in the pool.
	// If MaxIdle is 0 or less, no idle connections are kept. Put will close connections instead of idling them.
	MaxIdle int

	// MaxIdleTime is the maximum duration a connection can remain idle (unused)
	// in the pool before being closed by the background cleanup goroutine.
	// If MaxIdleTime is 0 or less, connections are not closed based on idle time.
	MaxIdleTime time.Duration

	// MaxAge is the maximum amount of time a connection may exist before being closed.
	// If MaxAge is 0 or less, connections are not closed based on their age.
	MaxAge time.Duration

	// CleanupInterval specifies the interval for the background cleanup goroutine
	// that closes connections based on MaxIdleTime and MaxAge.
	// If 0 or less, a default interval is calculated based on MaxIdleTime and HealthCheckInterval
	// (clamped between minCleanupInterval and maxCleanupInterval).
	CleanupInterval time.Duration

	// HealthCheckInterval is the interval at which idle connections are checked
	// for health using the Ping function. Requires Ping to be set.
	// If 0 or less, periodic background health checks are disabled.
	HealthCheckInterval time.Duration

	// HealthCheckTimeout specifies a time limit for the Ping function during
	// periodic health checks. This prevents a potentially slow or hanging Ping
	// call from blocking the pool's background cleanup goroutine indefinitely.
	// If a Ping call exceeds this timeout, the connection is considered
	// unhealthy and will be closed.
	// If 0 or less, no specific timeout is applied by the pool during these background checks
	// (the Ping function might still time out internally, or block indefinitely).
	HealthCheckTimeout time.Duration
}

Config defines the configuration for a connection pool. For time.Duration fields, a zero value typically disables the feature or uses a default. Negative values are generally treated as zero.

type Factory

type Factory func(ctx context.Context) (interface{}, error)

Factory defines a function that creates a new connection. The provided context can be used for timeouts or cancellation during connection creation. The returned connection should be ready to use.

type Metrics

type Metrics struct {
	// Total number of times Get was called.
	Gets uint64
	// Total number of times Put was called successfully (connection belonged to the pool).
	Puts uint64
	// Total number of times Discard was called.
	Discards uint64

	// Number of times Get successfully returned an existing idle connection (cache hit).
	Hits uint64
	// Number of times Get successfully created a new connection via the Factory (cache miss).
	Misses uint64
	// Number of times Get had to wait because the pool was full.
	Waits uint64
	// Number of times Get timed out or was cancelled by context while waiting for a connection.
	Timeouts uint64

	// Number of connections closed due to a failed Ping check (either in Get or background check).
	PingFailures uint64
	// Number of connections closed by the background cleanup goroutine due to exceeding MaxIdleTime.
	IdleClosed uint64

	// Number of connections closed by the background cleanup goroutine due to exceeding MaxAge.
	AgeClosed uint64
	// Number of times the Factory function returned an error during Get.
	FactoryFailures uint64
	// CloseTimeouts is currently unused as Close returns context errors directly.
	// (Reserved for potential future use if specific timeout tracking in Close is added).
	CloseTimeouts uint64

	// MaxActive records the high-water mark of active connections observed simultaneously.
	// This value is updated atomically when a new maximum is reached.
	MaxActive uint64
}

Metrics contains statistics about the pool's operations. All counters are updated atomically.

type PingFunc

type PingFunc func(conn interface{}) error

PingFunc defines a function that checks the health of a connection. It should return nil if the connection is healthy, or an error if it is not. PingFunc should not modify the connection state.

type Pool

type Pool interface {
	// Get retrieves a connection from the pool.
	// If the pool has an idle connection available, it may be returned.
	// If no idle connection is available, and the pool has not reached its
	// maximum size, a new connection may be created using the Factory.
	// If a PingFunc is configured, it may be called to check health before returning an existing connection.
	// If the pool is at maximum capacity and no idle connections are available,
	// Get will block until a connection is returned to the pool or the context
	// is cancelled.
	Get(ctx context.Context) (interface{}, error)

	// Put returns a connection to the pool, making it available for reuse.
	// If the connection is nil, invalid, or does not belong to this pool, Put attempts
	// to close it using the configured CloseFunc and returns.
	// If the pool is closed, the connection is closed.
	// Otherwise, Put attempts to hand the connection to a waiting goroutine (from Get).
	// If no goroutines are waiting, it adds the connection to the idle list, potentially
	// evicting the oldest idle connection if MaxIdle is reached.
	Put(conn interface{})

	// Discard removes a connection from the pool and closes it, usually because
	// the connection is known to be broken or unhealthy. Discard closes the connection
	// using the configured CloseFunc.
	// It returns ErrInvalidConnection if the provided connection is nil, does not belong
	// to this pool, or is already closed.
	// It returns ErrPoolClosed if the pool was already closed when Discard was called,
	// but still attempts to close the connection. Otherwise, it returns the result
	// of the underlying CloseFunc call.
	Discard(conn interface{}) error

	// Close closes the pool and all its connections.
	// After Close returns, Get will return an error.
	// Close immediately closes all idle connections. It then waits for all active
	// connections to be returned (via Put or Discard) before returning.
	// If the context is cancelled before all connections are returned, the context's
	// error is returned. Close returns ErrPoolClosed if called more than once.
	Close(ctx context.Context) error

	// Len returns the current number of connections in the pool (both active and idle).
	Len() int

	// IdleLen returns the current number of idle connections in the pool.
	IdleLen() int

	// Stats returns a snapshot of the pool's metrics.
	Stats() Metrics
}

Pool represents a generic connection pool. Implementations must be safe for concurrent use by multiple goroutines.

func NewPool

func NewPool(config Config) (Pool, error)

NewPool creates a new connection pool with the given configuration.

Directories

Path Synopsis
Package mock_connpool is a generated GoMock package.
Package mock_connpool is a generated GoMock package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL