Documentation
¶
Index ¶
- Constants
- Variables
- func CleanPath(path string, isS3 bool) string
- func ErrToChan(ctx context.Context, ch chan<- error, err error)
- func IsDirectory(prefix, fileName string) bool
- func PreSort(ctx context.Context, r reader, path string) error
- type Opt
- func WithAccessTier(tier string) Opt
- func WithChunkSize(bytes int) Opt
- func WithDir(path string) Opt
- func WithDirList(pathList []string) Opt
- func WithFile(path string) Opt
- func WithFileList(pathList []string) Opt
- func WithLogger(logger *slog.Logger) Opt
- func WithNestedDir() Opt
- func WithRemoveFiles() Opt
- func WithSkipDirCheck() Opt
- func WithSorting() Opt
- func WithStartAfter(v string) Opt
- func WithStorageClass(class string) Opt
- func WithUploadConcurrency(v int) Opt
- func WithValidator(v validator) Opt
- func WithWarmPollDuration(duration time.Duration) Opt
- type Options
Constants ¶
const DefaultPollWarmDuration = time.Minute
DefaultPollWarmDuration is the interval between requests to cloud providers, to get file status during files restore.
Variables ¶
var ( ErrEmptyStorage = errors.New("empty storage") ErrArchivedObject = errors.New("archived object") )
ErrEmptyStorage describes the empty storage error for the restore operation.
Functions ¶
func CleanPath ¶
CleanPath sanitizes the input path string based on the storage type (S3 or non-S3). For S3 storage, it removes the root path "/" as S3 uses empty string for root. For all storage types, it ensures proper trailing slash format except for empty or root paths. Returns the cleaned path string.
func ErrToChan ¶ added in v0.5.0
ErrToChan checks context before sending an error to errors chan. If context is already canceled and the reader must be stopped, no need to send error to errors chan.
func IsDirectory ¶
IsDirectory determines if a given file name represents a directory within the specified prefix. It considers three cases:
- File name ends with "/" (definite directory)
- File name is within a prefix and contains "/" after the prefix
- File name contains "/" (general case)
Returns true if the file name represents a directory, false otherwise.
func PreSort ¶
PreSort performs pre-processing of backup files by sorting them before reading. It retrieves a list of objects from the specified path, sorts them according to the backup file naming conventions, and configures the reader to stream the sorted list. Returns an error if listing objects fails or if the sorting operation fails.
Types ¶
type Opt ¶
type Opt func(*Options)
func WithAccessTier ¶
WithAccessTier sets the access tier for archived files. For readers, archived files are restored to the specified tier.
func WithChunkSize ¶
WithChunkSize sets the chunk size for uploading files. Is used only for Writer.
func WithDirList ¶
WithDirList adds the directory list to read files from. Is used only for Reader.
func WithFileList ¶
WithFileList adds the file list to read from. Is used only for Reader.
func WithNestedDir ¶
func WithNestedDir() Opt
WithNestedDir adds WithNestedDir = true parameter. That means that we won't skip nested folders.
func WithRemoveFiles ¶
func WithRemoveFiles() Opt
WithRemoveFiles adds remove files flag, so all files will be removed from backup folder before backup. Is used only for Writer.
func WithSkipDirCheck ¶
func WithSkipDirCheck() Opt
WithSkipDirCheck adds skip dir check flags. Which means that backup directory won't be checked for emptiness.
func WithSorting ¶
func WithSorting() Opt
WithSorting adds a sorting flag. Which means that files will be read from directory in the sorted order. Is used only for Reader.
func WithStartAfter ¶
WithStartAfter adds start after parameter to list request. Is used only for Reader.
func WithStorageClass ¶
WithStorageClass sets the storage class. For writers, files are saved to the specified class (in case of Azure it will be tier).
func WithUploadConcurrency ¶
WithUploadConcurrency defines max number of concurrent uploads to be performed to upload the file. Is used only for Azure Writer.
func WithValidator ¶
func WithValidator(v validator) Opt
WithValidator adds the Validator to Reader, so files will be validated before reading. Is used only for Reader.
func WithWarmPollDuration ¶
WithWarmPollDuration sets the restore status polling interval.
type Options ¶
type Options struct { // PathList contains a list of files or directories. PathList []string // IsDir flag determines whether the path is a directory. IsDir bool // IsRemovingFiles flag specifies whether to clear the backup folder. IsRemovingFiles bool // Validator contains files Validator that is applied to files if IsDir is true. Validator validator // WithNestedDir determines whether the read/write operations should treat objects // identified as directories like regular files. // When we stream files or delete files in folder, we skip directories. Setting // WithNestedDir to true disables this directory skipping behavior. // Default: false WithNestedDir bool // StartAfter is an artificial parameter. Used to skip objects in the storage. // The Result will not include an object specified in startAfter. // If it is set, then we compare the names received from the storage lexicographically, // and if the name is less than the specified parameter, we skip this object. StartAfter string // SkipDirCheck, if true, the backup directory won't be checked. SkipDirCheck bool // SortFiles determines whether we need to sort files before reading. SortFiles bool // UploadConcurrency defines the max number of concurrent uploads to be performed to // upload the file. Each concurrent upload will create a buffer of size BlockSize. UploadConcurrency int // StorageClass specifies the type of storage class. // Supported values depend on the cloud provider being used. StorageClass string // The access tier that will be applied to back up files. AccessTier string // Logger contains logger. Logger *slog.Logger // How often restore status will be polled from cloud provider. PollWarmDuration time.Duration // Size of chunk to upload. ChunkSize int }