filesystem

package
v6.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 2, 2025 License: GPL-3.0 Imports: 33 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	LayoutNameFlat    = "flat"
	LayoutNameByEpoch = "by-epoch"
)

Variables

View Source 
var LayoutNames = []string{LayoutNameFlat, LayoutNameByEpoch}

Functions

func NewEphemeralBlobStorageWithMocker 

func NewEphemeralBlobStorageWithMocker(t testing.TB) (*BlobMocker, *BlobStorage)

NewEphemeralBlobStorageWithMocker returns a *BlobMocker value in addition to the BlobStorage value. BlockMocker encapsulates things blob path construction to avoid leaking implementation details.

func NewEphemeralDataColumnStorageWithMocker added in v6.0.4 

func NewEphemeralDataColumnStorageWithMocker(t testing.TB) (*DataColumnMocker, *DataColumnStorage)

NewEphemeralDataColumnStorageWithMocker returns a *BlobMocker value in addition to the BlobStorage value. BlockMocker encapsulates things blob path construction to avoid leaking implementation details.

Types

type BlobMocker 

type BlobMocker struct {
	// contains filtered or unexported fields
}

func (*BlobMocker) CreateFakeIndices 

func (bm *BlobMocker) CreateFakeIndices(root [fieldparams.RootLength]byte, slot primitives.Slot, indices ...uint64) error

CreateFakeIndices creates empty blob sidecar files at the expected path for the given root and indices to influence the result of Indices().

type BlobStorage 

type BlobStorage struct {
	// contains filtered or unexported fields
}

BlobStorage is the concrete implementation of the filesystem backend for saving and retrieving BlobSidecars.

func NewBlobStorage 

func NewBlobStorage(opts ...BlobStorageOption) (*BlobStorage, error)

NewBlobStorage creates a new instance of the BlobStorage object. Note that the implementation of BlobStorage may attempt to hold a file lock to guarantee exclusive control of the blob storage directory, so this should only be initialized once per beacon node.

func NewEphemeralBlobStorage 

func NewEphemeralBlobStorage(t testing.TB, opts ...BlobStorageOption) *BlobStorage

NewEphemeralBlobStorage should only be used for tests. The instance of BlobStorage returned is backed by an in-memory virtual filesystem, improving test performance and simplifying cleanup.

func NewEphemeralBlobStorageAndFs 

func NewEphemeralBlobStorageAndFs(t testing.TB, opts ...BlobStorageOption) (afero.Fs, *BlobStorage)

NewEphemeralBlobStorageAndFs can be used by tests that want access to the virtual filesystem in order to interact with it outside the parameters of the BlobStorage api.

func NewEphemeralBlobStorageUsingFs 

func NewEphemeralBlobStorageUsingFs(t testing.TB, fs afero.Fs, opts ...BlobStorageOption) *BlobStorage

func NewWarmedEphemeralBlobStorageUsingFs 

func NewWarmedEphemeralBlobStorageUsingFs(t testing.TB, fs afero.Fs, opts ...BlobStorageOption) *BlobStorage

func (*BlobStorage) Clear 

func (bs *BlobStorage) Clear() error

Clear deletes all files on the filesystem.

func (*BlobStorage) Get 

func (bs *BlobStorage) Get(root [32]byte, idx uint64) (blocks.VerifiedROBlob, error)

Get retrieves a single BlobSidecar by its root and index. Since BlobStorage only writes blobs that have undergone full verification, the return value is always a VerifiedROBlob.

func (*BlobStorage) Remove 

func (bs *BlobStorage) Remove(root [32]byte) error

Remove removes all blobs for a given root.

func (*BlobStorage) Save 

func (bs *BlobStorage) Save(sidecar blocks.VerifiedROBlob) error

Save saves blobs given a list of sidecars.

func (*BlobStorage) Summary 

func (bs *BlobStorage) Summary(root [32]byte) BlobStorageSummary

Summary returns the BlobStorageSummary from the layout. Internally, this is a cached representation of the directory listing for the given root.

func (*BlobStorage) WarmCache 

func (bs *BlobStorage) WarmCache()

WarmCache runs the prune routine with an expiration of slot of 0, so nothing will be pruned, but the pruner's cache will be populated at node startup, avoiding a costly cold prune (~4s in syscalls) during syncing.

func (*BlobStorage) WithinRetentionPeriod 

func (bs *BlobStorage) WithinRetentionPeriod(requested, current primitives.Epoch) bool

WithinRetentionPeriod checks if the requested epoch is within the blob retention period.

type BlobStorageOption 

type BlobStorageOption func(*BlobStorage) error

BlobStorageOption is a functional option for configuring a BlobStorage.

func WithBasePath 

func WithBasePath(base string) BlobStorageOption

WithBasePath is a required option that sets the base path of blob storage.

func WithBlobRetentionEpochs 

func WithBlobRetentionEpochs(e primitives.Epoch) BlobStorageOption

WithBlobRetentionEpochs is an option that changes the number of epochs blobs will be persisted.

func WithFs 

func WithFs(fs afero.Fs) BlobStorageOption

WithFs allows the afero.Fs implementation to be customized. Used by tests to substitute an in-memory filesystem.

func WithLayout 

func WithLayout(name string) BlobStorageOption

WithLayout enables the user to specify which layout scheme to use, dictating how blob files are stored on disk.

func WithSaveFsync 

func WithSaveFsync(fsync bool) BlobStorageOption

WithSaveFsync is an option that causes Save to call fsync before renaming part files for improved durability.

type BlobStorageSummarizer 

type BlobStorageSummarizer interface {
	Summary(root [32]byte) BlobStorageSummary
}

BlobStorageSummarizer can be used to receive a summary of metadata about blobs on disk for a given root. The BlobStorageSummary can be used to check which indices (if any) are available for a given block by root.

func NewMockBlobStorageSummarizer 

func NewMockBlobStorageSummarizer(t *testing.T, set map[[32]byte][]int) BlobStorageSummarizer

type BlobStorageSummary 

type BlobStorageSummary struct {
	// contains filtered or unexported fields
}

BlobStorageSummary represents cached information about the BlobSidecars on disk for each root the cache knows about.

func NewBlobStorageSummary 

func NewBlobStorageSummary(epoch primitives.Epoch, mask []bool) (BlobStorageSummary, error)

NewBlobStorageSummary creates a new BlobStorageSummary for a given epoch and mask.

func (BlobStorageSummary) AllAvailable 

func (s BlobStorageSummary) AllAvailable(count int) bool

AllAvailable returns true if we have all blobs for all indices from 0 to count-1.

func (BlobStorageSummary) HasIndex 

func (s BlobStorageSummary) HasIndex(idx uint64) bool

HasIndex returns true if the BlobSidecar at the given index is available in the filesystem.

func (BlobStorageSummary) MaxBlobsForEpoch 

func (s BlobStorageSummary) MaxBlobsForEpoch() uint64

type DataColumnMocker added in v6.0.4 

type DataColumnMocker struct {
	// contains filtered or unexported fields
}

func (*DataColumnMocker) CreateFakeIndices added in v6.0.4 

func (bm *DataColumnMocker) CreateFakeIndices(root [fieldparams.RootLength]byte, slot primitives.Slot, indices ...uint64) error

CreateFakeIndices creates empty blob sidecar files at the expected path for the given root and indices to influence the result of Indices().

type DataColumnStorage added in v6.0.4 

type DataColumnStorage struct {
	// contains filtered or unexported fields
}

DataColumnStorage is the concrete implementation of the filesystem backend for saving and retrieving DataColumnSidecars.

func NewDataColumnStorage added in v6.0.4 

func NewDataColumnStorage(ctx context.Context, opts ...DataColumnStorageOption) (*DataColumnStorage, error)

NewDataColumnStorage creates a new instance of the DataColumnStorage object. Note that the implementation of DataColumnStorage may attempt to hold a file lock to guarantee exclusive control of the data column storage directory, so this should only be initialized once per beacon node.

func NewEphemeralDataColumnStorage added in v6.0.4 

func NewEphemeralDataColumnStorage(t testing.TB, opts ...DataColumnStorageOption) *DataColumnStorage

NewEphemeralDataColumnStorage should only be used for tests. The instance of DataColumnStorage returned is backed by an in-memory virtual filesystem, improving test performance and simplifying cleanup.

func NewEphemeralDataColumnStorageAndFs added in v6.0.4 

func NewEphemeralDataColumnStorageAndFs(t testing.TB, opts ...DataColumnStorageOption) (afero.Fs, *DataColumnStorage)

NewEphemeralDataColumnStorageAndFs can be used by tests that want access to the virtual filesystem in order to interact with it outside the parameters of the DataColumnStorage API.

func NewEphemeralDataColumnStorageUsingFs added in v6.0.4 

func NewEphemeralDataColumnStorageUsingFs(t testing.TB, fs afero.Fs, opts ...DataColumnStorageOption) *DataColumnStorage

func NewWarmedEphemeralDataColumnStorageUsingFs added in v6.0.4 

func NewWarmedEphemeralDataColumnStorageUsingFs(t testing.TB, fs afero.Fs, opts ...DataColumnStorageOption) *DataColumnStorage

func (*DataColumnStorage) Clear added in v6.0.4 

func (dcs *DataColumnStorage) Clear() error

Clear deletes all files on the filesystem.

func (*DataColumnStorage) Get added in v6.0.4 

func (dcs *DataColumnStorage) Get(root [fieldparams.RootLength]byte, indices []uint64) ([]blocks.VerifiedRODataColumn, error)

Get retrieves data column sidecars from the database. If one of the requested data column sidecars is not found, it is just skipped. If indices is nil, then all stored data column sidecars are returned. Since DataColumnStorage only writes data columns that have undergone full verification, the return value is always a VerifiedRODataColumn.

func (*DataColumnStorage) Remove added in v6.0.4 

func (dcs *DataColumnStorage) Remove(blockRoot [fieldparams.RootLength]byte) error

Remove deletes all data column sidecars for a given root.

func (*DataColumnStorage) Save added in v6.0.4 

func (dcs *DataColumnStorage) Save(dataColumnSidecars []blocks.VerifiedRODataColumn) error

Save saves data column sidecars into the database and asynchronously performs pruning. The returned chanel is closed when the pruning is complete.

func (*DataColumnStorage) Subscribe added in v6.0.4 

func (dcs *DataColumnStorage) Subscribe() (event.Subscription, <-chan DataColumnsIdent)

Subscribe subscribes to the data column feed. It returns the subscription and a 1-size buffer channel to receive data column sidecars. It is the responsibility of the caller to: - call `subscription.Unsubscribe` when done, and to - read from the channel as fast as possible until the channel is closed. A call to `Save` will buffer a new value to the channel. If a call to `Save` is done while a value is already in the buffer of the channel: - the call to `Save` will block until the new value can be bufferized in the channel, and - all other subscribers won't be notified until the new value can be bufferized in the channel.

func (*DataColumnStorage) Summary added in v6.0.4 

func (dcs *DataColumnStorage) Summary(root [fieldparams.RootLength]byte) DataColumnStorageSummary

Summary returns the DataColumnStorageSummary.

func (*DataColumnStorage) WarmCache added in v6.0.4 

func (dcs *DataColumnStorage) WarmCache()

WarmCache warms the cache of the data column filesystem. It holds the database (read) lock for all the time it is running.

type DataColumnStorageOption added in v6.0.4 

type DataColumnStorageOption func(*DataColumnStorage) error

DataColumnStorageOption is a functional option for configuring a DataColumnStorage.

func WithDataColumnBasePath added in v6.0.4 

func WithDataColumnBasePath(base string) DataColumnStorageOption

WithDataColumnBasePath is a required option that sets the base path of data column storage.

func WithDataColumnFs added in v6.0.4 

func WithDataColumnFs(fs afero.Fs) DataColumnStorageOption

WithDataColumnFs allows the afero.Fs implementation to be customized. Used by tests to substitute an in-memory filesystem.

func WithDataColumnRetentionEpochs added in v6.0.4 

func WithDataColumnRetentionEpochs(e primitives.Epoch) DataColumnStorageOption

WithDataColumnRetentionEpochs is an option that changes the number of epochs data columns will be persisted.

type DataColumnStorageSummarizer added in v6.0.4 

type DataColumnStorageSummarizer interface {
	Summary(root [fieldparams.RootLength]byte) DataColumnStorageSummary
}

DataColumnStorageSummarizer can be used to receive a summary of metadata about data columns on disk for a given root. The DataColumnStorageSummary can be used to check which indices (if any) are available for a given block by root.

func NewMockDataColumnStorageSummarizer added in v6.0.4 

func NewMockDataColumnStorageSummarizer(t *testing.T, set map[[fieldparams.RootLength]byte][]uint64) DataColumnStorageSummarizer

type DataColumnStorageSummary added in v6.0.4 

type DataColumnStorageSummary struct {
	// contains filtered or unexported fields
}

DataColumnStorageSummary represents cached information about the DataColumnSidecars on disk for each root the cache knows about.

func NewDataColumnStorageSummary added in v6.0.4 

func NewDataColumnStorageSummary(epoch primitives.Epoch, mask [fieldparams.NumberOfColumns]bool) DataColumnStorageSummary

NewDataColumnStorageSummary creates a new DataColumnStorageSummary for a given epoch and mask.

func (DataColumnStorageSummary) AllAvailable added in v6.0.4 

func (s DataColumnStorageSummary) AllAvailable(indices map[uint64]bool) bool

AllAvailable returns true if we have all data columns for corresponding indices.

func (DataColumnStorageSummary) Count added in v6.0.4 

func (s DataColumnStorageSummary) Count() uint64

Count returns the number of available data columns.

func (DataColumnStorageSummary) HasAtLeastOneIndex added in v6.0.4 

func (s DataColumnStorageSummary) HasAtLeastOneIndex(indices []uint64) bool

HasAtLeastOneIndex returns true if at least one of the DataColumnSidecars at the given indices is available in the filesystem.

func (DataColumnStorageSummary) HasIndex added in v6.0.4 

func (s DataColumnStorageSummary) HasIndex(index uint64) bool

HasIndex returns true if the DataColumnSidecar at the given index is available in the filesystem.

func (DataColumnStorageSummary) Stored added in v6.0.4 

func (s DataColumnStorageSummary) Stored() map[uint64]bool

Stored returns a map of all stored data columns.

type DataColumnsIdent added in v6.0.4 

type DataColumnsIdent struct {
	Root    [fieldparams.RootLength]byte
	Epoch   primitives.Epoch
	Indices []uint64
}

DataColumnsIdent is a collection of unique identifiers for data column sidecars.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.