Documentation
¶
Overview ¶
Package encoding provides utilities for encoding and decoding data objects.
Index ¶
- Variables
- func LogsDataset(dec LogsDecoder, sec *filemd.SectionInfo) dataset.Dataset
- func StreamsDataset(dec StreamsDecoder, sec *filemd.SectionInfo) dataset.Dataset
- type Decoder
- type Encoder
- type LogsColumnEncoder
- type LogsDecoder
- type LogsEncoder
- type Metrics
- type StreamsColumnEncoder
- type StreamsDecoder
- type StreamsEncoder
Constants ¶
This section is empty.
Variables ¶
var ( ErrElementExist = errors.New("open element already exists") ErrClosed = errors.New("element is closed") )
Functions ¶
func LogsDataset ¶
func LogsDataset(dec LogsDecoder, sec *filemd.SectionInfo) dataset.Dataset
LogsDataset implements returns a dataset.Dataset from a LogsDecoder for the given section.
func StreamsDataset ¶
func StreamsDataset(dec StreamsDecoder, sec *filemd.SectionInfo) dataset.Dataset
StreamsDataset implements returns a dataset.Dataset from a StreamsDecoder for the given section.
Types ¶
type Decoder ¶
type Decoder interface {
// Sections returns the list of sections within a data object.
Sections(ctx context.Context) ([]*filemd.SectionInfo, error)
// StreamsDecoder returns a decoder for streams sections.
StreamsDecoder() StreamsDecoder
// LogsDecoder returns a decoder for logs sections.
LogsDecoder() LogsDecoder
}
A Decoder decodes a data object.
func BucketDecoder ¶
func BucketDecoder(bucket objstore.BucketReader, path string) Decoder
BucketDecoder decodes a data object from the provided path within the specified objstore.BucketReader.
func ReaderAtDecoder ¶ added in v3.5.0
ReaderAtDecoder decodes a data object from the provided io.ReaderAt. The size argument specifies the total number of bytes in r.
type Encoder ¶
type Encoder struct {
// contains filtered or unexported fields
}
Encoder encodes a data object. Data objects are hierarchical, split into distinct sections that contain their own hierarchy.
To support hierarchical encoding, a set of Open* methods are provided to open a child element. Only one child element may be open at a given time; call Commit or Discard on a child element to close it.
func NewEncoder ¶
NewEncoder creates a new Encoder which writes a data object to the provided writer.
func (*Encoder) Flush ¶
Flush flushes any buffered data to the underlying writer. After flushing, enc is reset. Flush fails if there is currently an open section.
func (*Encoder) MetadataSize ¶
MetadataSize returns an estimate of the current size of the metadata for the data object. MetadataSize does not include the size of data appended. The estimate includes the currently open element.
func (*Encoder) OpenLogs ¶
func (enc *Encoder) OpenLogs() (*LogsEncoder, error)
OpenLogs opens a LogsEncoder. OpenLogs fails if there is another open section.
func (*Encoder) OpenStreams ¶
func (enc *Encoder) OpenStreams() (*StreamsEncoder, error)
OpenStreams opens a StreamsEncoder. OpenStreams fails if there is another open section.
type LogsColumnEncoder ¶
type LogsColumnEncoder struct {
// contains filtered or unexported fields
}
LogsColumnEncoder encodes an individual column in a logs section. LogsColumnEncoder are created by LogsEncoder.
func (*LogsColumnEncoder) AppendPage ¶
func (enc *LogsColumnEncoder) AppendPage(page *dataset.MemPage) error
AppendPage appens a new dataset.MemPage to the column. AppendPage fails if the column has been closed.
func (*LogsColumnEncoder) Commit ¶
func (enc *LogsColumnEncoder) Commit() error
Commit closes the column, flushing all data to the parent element. After Commit is called, the LogsColumnEncoder can no longer be modified.
func (*LogsColumnEncoder) Discard ¶
func (enc *LogsColumnEncoder) Discard() error
Discard discards the column, discarding any data written to it. After Discard is called, the LogsColumnEncoder can no longer be modified.
func (*LogsColumnEncoder) MetadataSize ¶
func (enc *LogsColumnEncoder) MetadataSize() int
MetadataSize returns an estimate of the current size of the metadata for the column. MetadataSize does not include the size of data appended.
type LogsDecoder ¶
type LogsDecoder interface {
// Columns describes the set of columns in the provided section.
Columns(ctx context.Context, section *filemd.SectionInfo) ([]*logsmd.ColumnDesc, error)
// Pages retrieves the set of pages for the provided columns. The order of
// page lists emitted by the sequence matches the order of columns
// provided: the first page list corresponds to the first column, and so
// on.
Pages(ctx context.Context, columns []*logsmd.ColumnDesc) result.Seq[[]*logsmd.PageDesc]
// ReadPages reads the provided set of pages, iterating over their data
// matching the argument order. If an error is encountered while retrieving
// pages, an error is emitted and iteration stops.
ReadPages(ctx context.Context, pages []*logsmd.PageDesc) result.Seq[dataset.PageData]
}
LogsDecoder supports decoding data within a logs section.
type LogsEncoder ¶
type LogsEncoder struct {
// contains filtered or unexported fields
}
LogsEncoder encodes an individual logs section in a data object. LogsEncoders are created by [Encoder]s.
func (*LogsEncoder) Commit ¶
func (enc *LogsEncoder) Commit() error
Commit closes the section, flushing all data to the parent element. After Commit is called, the LogsEncoder can no longer be modified.
Commit fails if there is an open column.
func (*LogsEncoder) Discard ¶
func (enc *LogsEncoder) Discard() error
Discard discards the section, discarding any data written to it. After Discard is called, the LogsEncoder can no longer be modified.
Discard fails if there is an open column.
func (*LogsEncoder) MetadataSize ¶
func (enc *LogsEncoder) MetadataSize() int
MetadataSize returns an estimate of the current size of the metadata for the section. MetadataSize includes an estimate for the currently open element.
func (*LogsEncoder) OpenColumn ¶
func (enc *LogsEncoder) OpenColumn(columnType logsmd.ColumnType, info *dataset.ColumnInfo) (*LogsColumnEncoder, error)
OpenColumn opens a new column in the logs section. OpenColumn fails if there is another open column or if the LogsEncoder has been closed.
type Metrics ¶
type Metrics struct {
// contains filtered or unexported fields
}
Metrics instruments encoded data objects.
func (*Metrics) Register ¶
func (m *Metrics) Register(reg prometheus.Registerer) error
Register registers metrics to report to reg.
func (*Metrics) Unregister ¶
func (m *Metrics) Unregister(reg prometheus.Registerer)
Unregister unregisters metrics from the provided Registerer.
type StreamsColumnEncoder ¶
type StreamsColumnEncoder struct {
// contains filtered or unexported fields
}
StreamsColumnEncoder encodes an individual column in a streams section. StreamsColumnEncoder are created by StreamsEncoder.
func (*StreamsColumnEncoder) AppendPage ¶
func (enc *StreamsColumnEncoder) AppendPage(page *dataset.MemPage) error
AppendPage appens a new dataset.MemPage to the column. AppendPage fails if the column has been closed.
func (*StreamsColumnEncoder) Commit ¶
func (enc *StreamsColumnEncoder) Commit() error
Commit closes the column, flushing all data to the parent element. After Commit is called, the StreamsColumnEncoder can no longer be modified.
func (*StreamsColumnEncoder) Discard ¶
func (enc *StreamsColumnEncoder) Discard() error
Discard discards the column, discarding any data written to it. After Discard is called, the StreamsColumnEncoder can no longer be modified.
func (*StreamsColumnEncoder) MetadataSize ¶
func (enc *StreamsColumnEncoder) MetadataSize() int
MetadataSize returns an estimate of the current size of the metadata for the column. MetadataSize does not include the size of data appended.
type StreamsDecoder ¶
type StreamsDecoder interface {
// Columns describes the set of columns in the provided section.
Columns(ctx context.Context, section *filemd.SectionInfo) ([]*streamsmd.ColumnDesc, error)
// Pages retrieves the set of pages for the provided columns. The order of
// page lists emitted by the sequence matches the order of columns
// provided: the first page list corresponds to the first column, and so
// on.
Pages(ctx context.Context, columns []*streamsmd.ColumnDesc) result.Seq[[]*streamsmd.PageDesc]
// ReadPages reads the provided set of pages, iterating over their data
// matching the argument order. If an error is encountered while retrieving
// pages, an error is emitted and iteration stops.
ReadPages(ctx context.Context, pages []*streamsmd.PageDesc) result.Seq[dataset.PageData]
}
StreamsDecoder supports decoding data within a streams section.
type StreamsEncoder ¶
type StreamsEncoder struct {
// contains filtered or unexported fields
}
StreamsEncoder encodes an individual streams section in a data object. StreamsEncoder are created by [Encoder]s.
func (*StreamsEncoder) Commit ¶
func (enc *StreamsEncoder) Commit() error
Commit closes the section, flushing all data to the parent element. After Commit is called, the StreamsEncoder can no longer be modified.
Commit fails if there is an open column.
func (*StreamsEncoder) Discard ¶
func (enc *StreamsEncoder) Discard() error
Discard discards the section, discarding any data written to it. After Discard is called, the StreamsEncoder can no longer be modified.
Discard fails if there is an open column.
func (*StreamsEncoder) MetadataSize ¶
func (enc *StreamsEncoder) MetadataSize() int
MetadataSize returns an estimate of the current size of the metadata for the stream. MetadataSize includes an estimate for the currently open element.
func (*StreamsEncoder) OpenColumn ¶
func (enc *StreamsEncoder) OpenColumn(columnType streamsmd.ColumnType, info *dataset.ColumnInfo) (*StreamsColumnEncoder, error)
OpenColumn opens a new column in the streams section. OpenColumn fails if there is another open column or if the StreamsEncoder has been closed.
Source Files