sharded

package
v0.0.0-...-164552c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 14, 2025 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Log2Fixed64

func Log2Fixed64(in uint64) uint64

Log2Fixed64 computes log2(x) using fixed-point arithmetic. As the input is a 64-bit unsigned integer, this function yields a value between [0, 64), shifted by 64-log2(64)=58 bits to the left.

func NewShardedDownloader

func NewShardedDownloader[TReference object.BasicReference](shards []object.Downloader[TReference], shardNames []string, picker Picker) object.Downloader[TReference]

NewShardedDownloader creates a decorator for one or more object.Downloaders that spreads out incoming requests based on the provided reference.

func NewShardedUploader

func NewShardedUploader[TReference object.BasicReference, TLease any](shards []object.Uploader[TReference, TLease], shardNames []string, picker Picker) object.Uploader[TReference, TLease]

NewShardedUploader creates a decorator for one or more object.Uploaders that spreads out incoming requests based on the provided reference.

Types

type Picker

type Picker interface {
	PickShard(data []byte) int
}

Picker of shards. Based on opaque data, Picker computes a hash and uses it to select a backend with a uniform distribution.

func NewWeightedRendezvousPicker

func NewWeightedRendezvousPicker(weightedShards []WeightedShard) Picker

NewWeightedRendezvousPicker creates a picker of shards that uses rendezvous hashing. This ensures that if backends are added or removed, the key space is only affected minimally.

type WeightedShard

type WeightedShard struct {
	Key    []byte
	Weight uint32
}

WeightedShard describes the properties of a single shard that may be selected by WeightedRendezvousPicker.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL