lrucache

package
v1.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 6, 2025 License: MIT Imports: 9 Imported by: 3

README

LRUCache

GoDoc Widget

The lrucache package provides an in-memory cache with an LRU (Least Recently Used) eviction policy and Prometheus metrics integration.

Features

  • LRU Eviction Policy: Automatically removes the least recently used items when the cache reaches its maximum size.
  • Prometheus Metrics: Collects and exposes metrics to monitor cache usage and performance.
  • Expiration: Supports setting TTL (Time To Live) for entries. Expired entries are removed during cleanup or when accessed.
  • Cache Stampede Mitigation: Prevents multiple goroutines from loading the same key concurrently by using a single flight pattern.

Usage

Basic Example
package lrucache_test

import (
	"fmt"
	"log"

	"github.com/prometheus/client_golang/prometheus"

	"github.com/acronis/go-appkit/lrucache"
)

type User struct {
	UUID string
	Name string
}

type Post struct {
	UUID string
	Text string
}

func Example() {
	// Make and register Prometheus metrics collector.
	promMetrics := lrucache.NewPrometheusMetricsWithOpts(lrucache.PrometheusMetricsOpts{
		Namespace:         "my_app",                                  // Will be prepended to all metric names.
		ConstLabels:       prometheus.Labels{"app_version": "1.2.3"}, // Will be applied to all metrics.
		CurriedLabelNames: []string{"entry_type"},                    // For distinguishing between cached entities.
	})
	promMetrics.MustRegister()

	// LRU cache for users.
	const aliceUUID = "966971df-a592-4e7e-a309-52501016fa44"
	const bobUUID = "848adf28-84c1-4259-97a2-acba7cf5c0b6"
	usersCache, err := lrucache.New[string, User](100_000,
		promMetrics.MustCurryWith(prometheus.Labels{"entry_type": "user"}))
	if err != nil {
		log.Fatal(err)
	}
	usersCache.Add(aliceUUID, User{aliceUUID, "Alice"})
	usersCache.Add(bobUUID, User{bobUUID, "Bob"})
	if user, found := usersCache.Get(aliceUUID); found {
		fmt.Printf("User: %s, %s\n", user.UUID, user.Name)
	}
	if user, found := usersCache.Get(bobUUID); found {
		fmt.Printf("User: %s, %s\n", user.UUID, user.Name)
	}

	// LRU cache for posts. Posts are loaded from DB if not found in cache.
	const post1UUID = "823e50c7-984d-4de3-8a09-92fa21d3cc3b"
	const post2UUID = "24707009-ddf6-4e88-bd51-84ae236b7fda"
	postsCache, err := lrucache.NewWithOpts[string, Post](1_000,
		promMetrics.MustCurryWith(prometheus.Labels{"entry_type": "note"}), lrucache.Options{
			DefaultTTL: 5 * time.Minute, // Expired entries are removed during cleanup (see RunPeriodicCleanup method) or when accessed.
		})
	if err != nil {
		log.Fatal(err)
	}

	cleanupCtx, cleanupCancel := context.WithCancel(context.Background())
	defer cleanupCancel()
	go postsCache.RunPeriodicCleanup(cleanupCtx, 10*time.Minute) // Run cleanup every 10 minutes.

	loadPostFromDatabase := func(id string) (value Post, err error) {
		// Emulate loading post from DB.
		if id == post1UUID {
			return Post{id, "Lorem ipsum dolor sit amet..."}, nil
		}
		return Post{}, fmt.Errorf("not found")
	}

	for _, postID := range []string{post1UUID, post1UUID, post2UUID} {
		// Get post from cache or load it from DB. If two goroutines try to load the same post concurrently,
		// only one of them will actually load the post, while the other will wait for the first one to finish.
		if post, exists, loadErr := postsCache.GetOrLoad(postID, loadPostFromDatabase); loadErr != nil {
			fmt.Printf("Failed to load post %s: %v\n", postID, loadErr)
		} else {
			if exists {
				fmt.Printf("Post: %s, %s\n", post.UUID, post.Text)
			} else {
				fmt.Printf("Post (loaded from db): %s, %s\n", post.UUID, post.Text)
			}
		}
	}

	// The following Prometheus metrics will be exposed:
	// my_app_cache_entries_amount{app_version="1.2.3",entry_type="note"} 1
	// my_app_cache_entries_amount{app_version="1.2.3",entry_type="user"} 2
	// my_app_cache_hits_total{app_version="1.2.3",entry_type="note"} 1
	// my_app_cache_hits_total{app_version="1.2.3",entry_type="user"} 2
	// my_app_cache_misses_total{app_version="1.2.3",entry_type="note"} 2

	fmt.Printf("Users: %d\n", usersCache.Len())
	fmt.Printf("Posts: %d\n", postsCache.Len())

	// Output:
	// User: 966971df-a592-4e7e-a309-52501016fa44, Alice
	// User: 848adf28-84c1-4259-97a2-acba7cf5c0b6, Bob
	// Post (loaded from db): 823e50c7-984d-4de3-8a09-92fa21d3cc3b, Lorem ipsum dolor sit amet...
	// Post: 823e50c7-984d-4de3-8a09-92fa21d3cc3b, Lorem ipsum dolor sit amet...
	// Failed to load post 24707009-ddf6-4e88-bd51-84ae236b7fda: not found
	// Users: 2
	// Posts: 1
}
Prometheus Metrics

Here is the full list of Prometheus metrics exposed by the lrucache package:

  • cache_entries_amount: Total number of entries in the cache.
  • cache_hits_total: Number of successfully found keys in the cache.
  • cache_misses_total: Number of not found keys in the cache.
  • cache_evictions_total: Number of evicted entries.

These metrics can be further customized with namespaces, constant labels, and curried labels as shown in the examples.

License

Copyright © 2024 Acronis International GmbH.

Licensed under MIT License.

Documentation

Overview

Package lrucache provides in-memory cache with LRU eviction policy, expiration mechanism, and Prometheus metrics.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/prometheus/client_golang/prometheus"

	"github.com/acronis/go-appkit/lrucache"
)

type User struct {
	UUID string
	Name string
}

type Post struct {
	UUID string
	Text string
}

func main() {
	// Make and register Prometheus metrics collector.
	promMetrics := lrucache.NewPrometheusMetricsWithOpts(lrucache.PrometheusMetricsOpts{
		Namespace:         "my_app",                                  // Will be prepended to all metric names.
		ConstLabels:       prometheus.Labels{"app_version": "1.2.3"}, // Will be applied to all metrics.
		CurriedLabelNames: []string{"entry_type"},                    // For distinguishing between cached entities.
	})
	promMetrics.MustRegister()

	// LRU cache for users.
	const aliceUUID = "966971df-a592-4e7e-a309-52501016fa44"
	const bobUUID = "848adf28-84c1-4259-97a2-acba7cf5c0b6"
	usersCache, err := lrucache.New[string, User](100_000,
		promMetrics.MustCurryWith(prometheus.Labels{"entry_type": "user"}))
	if err != nil {
		log.Fatal(err)
	}
	usersCache.Add(aliceUUID, User{aliceUUID, "Alice"})
	usersCache.Add(bobUUID, User{bobUUID, "Bob"})
	if user, found := usersCache.Get(aliceUUID); found {
		fmt.Printf("User: %s, %s\n", user.UUID, user.Name)
	}
	if user, found := usersCache.Get(bobUUID); found {
		fmt.Printf("User: %s, %s\n", user.UUID, user.Name)
	}

	// LRU cache for posts. Posts are loaded from DB if not found in cache.
	const post1UUID = "823e50c7-984d-4de3-8a09-92fa21d3cc3b"
	const post2UUID = "24707009-ddf6-4e88-bd51-84ae236b7fda"
	postsCache, err := lrucache.NewWithOpts[string, Post](1_000,
		promMetrics.MustCurryWith(prometheus.Labels{"entry_type": "note"}), lrucache.Options{
			DefaultTTL: 5 * time.Minute, // Expired entries are removed during cleanup (see RunPeriodicCleanup method) or when accessed.
		})
	if err != nil {
		log.Fatal(err)
	}

	cleanupCtx, cleanupCancel := context.WithCancel(context.Background())
	defer cleanupCancel()
	go postsCache.RunPeriodicCleanup(cleanupCtx, 10*time.Minute) // Run cleanup every 10 minutes.

	loadPostFromDatabase := func(id string) (value Post, err error) {
		// Emulate loading post from DB.
		if id == post1UUID {
			return Post{id, "Lorem ipsum dolor sit amet..."}, nil
		}
		return Post{}, fmt.Errorf("not found")
	}

	for _, postID := range []string{post1UUID, post1UUID, post2UUID} {
		// Get post from cache or load it from DB. If two goroutines try to load the same post concurrently,
		// only one of them will actually load the post, while the other will wait for the first one to finish.
		if post, exists, loadErr := postsCache.GetOrLoad(postID, loadPostFromDatabase); loadErr != nil {
			fmt.Printf("Failed to load post %s: %v\n", postID, loadErr)
		} else {
			if exists {
				fmt.Printf("Post: %s, %s\n", post.UUID, post.Text)
			} else {
				fmt.Printf("Post (loaded from db): %s, %s\n", post.UUID, post.Text)
			}
		}
	}

	// The following Prometheus metrics will be exposed:
	// my_app_cache_entries_amount{app_version="1.2.3",entry_type="note"} 1
	// my_app_cache_entries_amount{app_version="1.2.3",entry_type="user"} 2
	// my_app_cache_hits_total{app_version="1.2.3",entry_type="note"} 1
	// my_app_cache_hits_total{app_version="1.2.3",entry_type="user"} 2
	// my_app_cache_misses_total{app_version="1.2.3",entry_type="note"} 2

	fmt.Printf("Users: %d\n", usersCache.Len())
	fmt.Printf("Posts: %d\n", postsCache.Len())

}

Output:

User: 966971df-a592-4e7e-a309-52501016fa44, Alice
User: 848adf28-84c1-4259-97a2-acba7cf5c0b6, Bob
Post (loaded from db): 823e50c7-984d-4de3-8a09-92fa21d3cc3b, Lorem ipsum dolor sit amet...
Post: 823e50c7-984d-4de3-8a09-92fa21d3cc3b, Lorem ipsum dolor sit amet...
Failed to load post 24707009-ddf6-4e88-bd51-84ae236b7fda: not found
Users: 2
Posts: 1

Index

Examples

Constants

This section is empty.

Variables

View Source 
var ErrGoexit = errors.New("runtime.Goexit was called")

ErrGoexit is returned when a goroutine calls runtime.Goexit.

Functions

This section is empty.

Types

type LRUCache 

type LRUCache[K comparable, V any] struct {
	// contains filtered or unexported fields
}

LRUCache represents an LRU cache with eviction mechanism and Prometheus metrics.

func New 

func New[K comparable, V any](maxEntries int, metricsCollector MetricsCollector) (*LRUCache[K, V], error)

New creates a new LRUCache with the provided maximum number of entries and metrics collector.

func NewWithOpts added in v1.10.0 

func NewWithOpts[K comparable, V any](maxEntries int, metricsCollector MetricsCollector, opts Options) (*LRUCache[K, V], error)

NewWithOpts creates a new LRUCache with the provided maximum number of entries, metrics collector, and options. Metrics collector is used to collect statistics about cache usage. It can be nil, in this case, metrics will be disabled.

func (*LRUCache[K, V]) Add 

func (c *LRUCache[K, V]) Add(key K, value V)

Add adds a value to the cache with the provided key and type. If the cache is full, the oldest entry will be removed.

func (*LRUCache[K, V]) AddWithTTL added in v1.10.0 

func (c *LRUCache[K, V]) AddWithTTL(key K, value V, ttl time.Duration)

AddWithTTL adds a value to the cache with the provided key, type, and TTL. If the cache is full, the oldest entry will be removed. Please note that expired entries are not removed immediately, but only when they are accessed or during periodic cleanup (see RunPeriodicCleanup). If the TTL is less than or equal to 0, the value will not expire.

func (*LRUCache[K, V]) Get 

func (c *LRUCache[K, V]) Get(key K) (value V, ok bool)

Get returns a value from the cache by the provided key and type.

func (*LRUCache[K, V]) GetOrAdd added in v1.3.0 

func (c *LRUCache[K, V]) GetOrAdd(key K, valueProvider func() V) (value V, exists bool)

GetOrAdd returns a value from the cache by the provided key, and adds a new value with the default TTL if the key does not exist. The new value is provided by the valueProvider function, which is called only if the key does not exist. Note that the function is called under the LRUCache lock, so it should be fast and non-blocking. If you need to perform a blocking operation, consider using GetOrLoad instead.

func (*LRUCache[K, V]) GetOrAddWithTTL added in v1.10.0 

func (c *LRUCache[K, V]) GetOrAddWithTTL(key K, valueProvider func() V, ttl time.Duration) (value V, exists bool)

GetOrAddWithTTL returns a value from the cache by the provided key, and adds a new value with the specified TTL if the key does not exist. The new value is provided by the valueProvider function, which is called only if the key does not exist. Note that the function is called under the LRUCache lock, so it should be fast and non-blocking. If you need to perform a blocking operation, consider using GetOrLoadWithTTL instead. If the TTL is less than or equal to 0, the value will not expire.

func (*LRUCache[K, V]) GetOrLoad added in v1.15.0 

func (c *LRUCache[K, V]) GetOrLoad(
	key K, loadValue func(K) (value V, err error),
) (value V, exists bool, err error)

GetOrLoad returns a value from the cache by the provided key, and loads a new value if the key does not exist.

The new value is provided by the loadValue function, which is called only if the key does not exist. The loadValue function returns the value and error. If the loadValue function returns an error, the value will not be added to the cache.

Single flight pattern is used to prevent multiple concurrent calls for the same key. If executing goroutine panics, other goroutines will receive PanicError. PanicError contains the original panic value and stack trace. If executing goroutine calls runtime.Goexit, other goroutines will receive ErrGoexit.

func (*LRUCache[K, V]) GetOrLoadWithTTL added in v1.15.0 

func (c *LRUCache[K, V]) GetOrLoadWithTTL(
	key K, loadValue func(K) (value V, ttl time.Duration, err error),
) (value V, exists bool, err error)

GetOrLoadWithTTL returns a value from the cache by the provided key, and loads a new value if the key does not exist.

The new value is provided by the loadValue function, which is called only if the key does not exist. The loadValue function returns the value, TTL, and error. If the TTL is less than or equal to 0, the value will not expire. If the loadValue function returns an error, the value will not be added to the cache.

Single flight pattern is used to prevent multiple concurrent calls for the same key. If executing goroutine panics, other goroutines will receive PanicError. PanicError contains the original panic value and stack trace. If executing goroutine calls runtime.Goexit, other goroutines will receive ErrGoexit.

func (*LRUCache[K, V]) Len 

func (c *LRUCache[K, V]) Len() int

Len returns the number of items in the cache.

func (*LRUCache[K, V]) Purge 

func (c *LRUCache[K, V]) Purge()

Purge clears the cache. Keep in mind that this method does not reset the cache size and does not reset Prometheus metrics except for the total number of entries. All removed entries will not be counted as evictions.

func (*LRUCache[K, V]) Remove 

func (c *LRUCache[K, V]) Remove(key K) bool

Remove removes a value from the cache by the provided key and type.

func (*LRUCache[K, V]) Resize 

func (c *LRUCache[K, V]) Resize(size int) (evicted int)

Resize changes the cache size and returns the number of evicted entries.

func (*LRUCache[K, V]) RunPeriodicCleanup added in v1.10.0 

func (c *LRUCache[K, V]) RunPeriodicCleanup(ctx context.Context, cleanupInterval time.Duration)

RunPeriodicCleanup runs a cycle of periodic cleanup of expired entries. Entries without expiration time are not affected. It's supposed to be run in a separate goroutine.

type MetricsCollector 

type MetricsCollector interface {
	// SetAmount sets the total number of entries in the cache.
	SetAmount(int)

	// IncHits increments the total number of successfully found keys in the cache.
	IncHits()

	// IncMisses increments the total number of not found keys in the cache.
	IncMisses()

	// AddEvictions increments the total number of evicted entries.
	AddEvictions(int)
}

MetricsCollector represents a collector of metrics to analyze how (effectively or not) cache is used.

type Options added in v1.10.0 

type Options struct {
	// DefaultTTL is the default TTL for the cache entries.
	// Please note that expired entries are not removed immediately,
	// but only when they are accessed or during periodic cleanup (see RunPeriodicCleanup).
	DefaultTTL time.Duration
}

Options represents options for the cache.

type PanicError added in v1.15.0 

type PanicError struct {
	Value interface{}
	Stack []byte
}

PanicError is an error that represents a panic value and stack trace.

func (*PanicError) Error added in v1.15.0 

func (p *PanicError) Error() string

func (*PanicError) Unwrap added in v1.15.0 

func (p *PanicError) Unwrap() error

type PrometheusMetrics added in v1.3.0 

type PrometheusMetrics struct {
	EntriesAmount  *prometheus.GaugeVec
	HitsTotal      *prometheus.CounterVec
	MissesTotal    *prometheus.CounterVec
	EvictionsTotal *prometheus.CounterVec
}

PrometheusMetrics represents a Prometheus metrics for the cache.

func NewPrometheusMetrics added in v1.3.0 

func NewPrometheusMetrics() *PrometheusMetrics

NewPrometheusMetrics creates a new instance of PrometheusMetrics with default options.

func NewPrometheusMetricsWithOpts added in v1.3.0 

func NewPrometheusMetricsWithOpts(opts PrometheusMetricsOpts) *PrometheusMetrics

NewPrometheusMetricsWithOpts creates a new instance of PrometheusMetrics with the provided options.

func (*PrometheusMetrics) AddEvictions added in v1.3.0 

func (pm *PrometheusMetrics) AddEvictions(n int)

AddEvictions increments the total number of evicted entries.

func (*PrometheusMetrics) IncHits added in v1.3.0 

func (pm *PrometheusMetrics) IncHits()

IncHits increments the total number of successfully found keys in the cache.

func (*PrometheusMetrics) IncMisses added in v1.3.0 

func (pm *PrometheusMetrics) IncMisses()

IncMisses increments the total number of not found keys in the cache.

func (*PrometheusMetrics) MustCurryWith added in v1.3.0 

func (pm *PrometheusMetrics) MustCurryWith(labels prometheus.Labels) *PrometheusMetrics

MustCurryWith curries the metrics collector with the provided labels.

func (*PrometheusMetrics) MustRegister added in v1.3.0 

func (pm *PrometheusMetrics) MustRegister()

MustRegister does registration of metrics collector in Prometheus and panics if any error occurs.

func (*PrometheusMetrics) SetAmount added in v1.3.0 

func (pm *PrometheusMetrics) SetAmount(amount int)

SetAmount sets the total number of entries in the cache.

func (*PrometheusMetrics) Unregister added in v1.3.0 

func (pm *PrometheusMetrics) Unregister()

Unregister cancels registration of metrics collector in Prometheus.

type PrometheusMetricsOpts added in v1.3.0 

type PrometheusMetricsOpts struct {
	// Namespace is a namespace for metrics. It will be prepended to all metric names.
	Namespace string

	// ConstLabels is a set of labels that will be applied to all metrics.
	ConstLabels prometheus.Labels

	// CurriedLabelNames is a list of label names that will be curried with the provided labels.
	// See PrometheusMetrics.MustCurryWith method for more details.
	// Keep in mind that if this list is not empty,
	// PrometheusMetrics.MustCurryWith method must be called further with the same labels.
	// Otherwise, the collector will panic.
	CurriedLabelNames []string
}

PrometheusMetricsOpts represents options for PrometheusMetrics.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.