Documentation
¶
Overview ¶
Package archiver contains the code which reads files, splits them into chunks and saves the data to the repository.
Index ¶
- Constants
- func TestCreateFiles(t testing.TB, target string, dir TestDir)
- func TestEnsureFileContent(ctx context.Context, t testing.TB, repo restic.BlobLoader, filename string, ...)
- func TestEnsureFiles(t testing.TB, target string, dir TestDir)
- func TestEnsureSnapshot(t testing.TB, repo restic.Repository, snapshotID restic.ID, dir TestDir)
- func TestEnsureTree(ctx context.Context, t testing.TB, prefix string, repo restic.BlobLoader, ...)
- func TestSnapshot(t testing.TB, repo restic.Repository, path string, parent *restic.ID) *restic.Snapshot
- func TestWalkFiles(t testing.TB, target string, dir TestDir, fn TestWalkFunc)
- type Archiver
- type ChangeStats
- type ErrorFunc
- type ItemStats
- type Options
- type RejectByNameFunc
- type RejectFunc
- func RejectByDevice(samples []string, filesystem fs.FS) (RejectFunc, error)
- func RejectBySize(maxSize int64) (RejectFunc, error)
- func RejectCloudFiles(warnf func(msg string, args ...interface{})) (RejectFunc, error)
- func RejectIfPresent(excludeFileSpec string, warnf func(msg string, args ...interface{})) (RejectFunc, error)
- type ScanStats
- type Scanner
- type SelectByNameFunc
- type SelectFunc
- type SnapshotOptions
- type Summary
- type TestDir
- type TestFile
- type TestHardlink
- type TestSymlink
- type TestWalkFunc
- type ToNoder
Constants ¶
const ( ChangeIgnoreCtime = 1 << iota ChangeIgnoreInode )
Flags for the ChangeIgnoreFlags bitfield.
Variables ¶
This section is empty.
Functions ¶
func TestCreateFiles ¶
TestCreateFiles creates a directory structure described by dir at target, which must already exist. On Windows, symlinks aren't created.
func TestEnsureFileContent ¶
func TestEnsureFileContent(ctx context.Context, t testing.TB, repo restic.BlobLoader, filename string, node *restic.Node, file TestFile)
TestEnsureFileContent checks if the file in the repo is the same as file.
func TestEnsureFiles ¶
TestEnsureFiles tests if the directory structure at target is the same as described in dir.
func TestEnsureSnapshot ¶
TestEnsureSnapshot tests if the snapshot in the repo has exactly the same structure as dir. On Windows, Symlinks are ignored.
func TestEnsureTree ¶
func TestEnsureTree(ctx context.Context, t testing.TB, prefix string, repo restic.BlobLoader, treeID restic.ID, dir TestDir)
TestEnsureTree checks that the tree ID in the repo matches dir. On Windows, Symlinks are ignored.
func TestSnapshot ¶
func TestSnapshot(t testing.TB, repo restic.Repository, path string, parent *restic.ID) *restic.Snapshot
TestSnapshot creates a new snapshot of path.
func TestWalkFiles ¶
func TestWalkFiles(t testing.TB, target string, dir TestDir, fn TestWalkFunc)
TestWalkFiles runs fn for each file/directory in dir, the filename will be constructed with target as the prefix. Symlinks on Windows are ignored.
Types ¶
type Archiver ¶
type Archiver struct { Repo archiverRepo SelectByName SelectByNameFunc Select SelectFunc FS fs.FS Options Options // Error is called for all errors that occur during backup. Error ErrorFunc // CompleteItem is called for all files and dirs once they have been // processed successfully. The parameter item contains the path as it will // be in the snapshot after saving. s contains some statistics about this // particular file/dir. // // Once reading a file has completed successfully (but not saving it yet), // CompleteItem will be called with current == nil. // // CompleteItem may be called asynchronously from several different // goroutines! CompleteItem func(item string, previous, current *restic.Node, s ItemStats, d time.Duration) // StartFile is called when a file is being processed by a worker. StartFile func(filename string) // CompleteBlob is called for all saved blobs for files. CompleteBlob func(bytes uint64) // WithAtime configures if the access time for files and directories should // be saved. Enabling it may result in much metadata, so it's off by // default. WithAtime bool // Flags controlling change detection. See doc/040_backup.rst for details. ChangeIgnoreFlags uint // contains filtered or unexported fields }
Archiver saves a directory structure to the repo.
An Archiver has a number of worker goroutines handling saving the different data structures to the repository, the details are implemented by the fileSaver, blobSaver, and treeSaver types.
The main goroutine (the one calling Snapshot()) traverses the directory tree and delegates all work to these worker pools. They return a futureNode which can be resolved later, by calling Wait() on it.
type ChangeStats ¶ added in v0.17.0
type ErrorFunc ¶
ErrorFunc is called when an error during archiving occurs. When nil is returned, the archiver continues, otherwise it aborts and passes the error up the call stack.
type ItemStats ¶
type ItemStats struct { DataBlobs int // number of new data blobs added for this item DataSize uint64 // sum of the sizes of all new data blobs DataSizeInRepo uint64 // sum of the bytes added to the repo (including compression and crypto overhead) TreeBlobs int // number of new tree blobs added for this item TreeSize uint64 // sum of the sizes of all new tree blobs TreeSizeInRepo uint64 // sum of the bytes added to the repo (including compression and crypto overhead) }
ItemStats collects some statistics about a particular file or directory.
type Options ¶
type Options struct { // ReadConcurrency sets how many files are read in concurrently. If // it's set to zero, at most two files are read in concurrently (which // turned out to be a good default for most situations). ReadConcurrency uint // SaveBlobConcurrency sets how many blobs are hashed and saved // concurrently. If it's set to zero, the default is the number of CPUs // available in the system. SaveBlobConcurrency uint // SaveTreeConcurrency sets how many trees are marshalled and saved to the // repo concurrently. SaveTreeConcurrency uint }
Options is used to configure the archiver.
func (Options) ApplyDefaults ¶
ApplyDefaults returns a copy of o with the default options set for all unset fields.
type RejectByNameFunc ¶ added in v0.18.0
RejectByNameFunc is a function that takes a filename of a file that would be included in the backup. The function returns true if it should be excluded (rejected) from the backup.
type RejectFunc ¶ added in v0.18.0
RejectFunc is a function that takes a filename and os.FileInfo of a file that would be included in the backup. The function returns true if it should be excluded (rejected) from the backup.
func RejectByDevice ¶ added in v0.18.0
func RejectByDevice(samples []string, filesystem fs.FS) (RejectFunc, error)
RejectByDevice returns a RejectFunc that rejects files which are on a different file systems than the files/dirs in samples.
func RejectBySize ¶ added in v0.18.0
func RejectBySize(maxSize int64) (RejectFunc, error)
func RejectCloudFiles ¶ added in v0.18.0
func RejectCloudFiles(warnf func(msg string, args ...interface{})) (RejectFunc, error)
RejectCloudFiles returns a func which rejects files which are online-only cloud files
func RejectIfPresent ¶ added in v0.18.0
func RejectIfPresent(excludeFileSpec string, warnf func(msg string, args ...interface{})) (RejectFunc, error)
RejectIfPresent returns a RejectByNameFunc which itself returns whether a path should be excluded. The RejectByNameFunc considers a file to be excluded when it resides in a directory with an exclusion file, that is specified by excludeFileSpec in the form "filename[:content]". The returned error is non-nil if the filename component of excludeFileSpec is empty. If rc is non-nil, it is going to be used in the RejectByNameFunc to expedite the evaluation of a directory based on previous visits.
type Scanner ¶
type Scanner struct { FS fs.FS SelectByName SelectByNameFunc Select SelectFunc Error ErrorFunc Result func(item string, s ScanStats) }
Scanner traverses the targets and calls the function Result with cumulated stats concerning the files and folders found. Select is used to decide which items should be included. Error is called when an error occurs.
type SelectByNameFunc ¶
SelectByNameFunc returns true for all items that should be included (files and dirs). If false is returned, files are ignored and dirs are not even walked.
func CombineRejectByNames ¶ added in v0.18.0
func CombineRejectByNames(funcs []RejectByNameFunc) SelectByNameFunc
type SelectFunc ¶
SelectFunc returns true for all items that should be included (files and dirs). If false is returned, files are ignored and dirs are not even walked.
func CombineRejects ¶ added in v0.18.0
func CombineRejects(funcs []RejectFunc) SelectFunc
type SnapshotOptions ¶
type SnapshotOptions struct { Tags restic.TagList Hostname string Excludes []string BackupStart time.Time Time time.Time ParentSnapshot *restic.Snapshot ProgramVersion string // SkipIfUnchanged omits the snapshot creation if it is identical to the parent snapshot. SkipIfUnchanged bool }
SnapshotOptions collect attributes for a new snapshot.
type TestDir ¶
type TestDir map[string]interface{}
TestDir describes a directory structure to create for a test.
type TestFile ¶
type TestFile struct {
Content string
}
TestFile describes a file created for a test.
type TestHardlink ¶ added in v0.17.0
type TestHardlink struct {
Target string
}
TestHardlink describes a hardlink created for a test.
func (TestHardlink) String ¶ added in v0.17.0
func (s TestHardlink) String() string
type TestSymlink ¶
type TestSymlink struct {
Target string
}
TestSymlink describes a symlink created for a test.
func (TestSymlink) String ¶
func (s TestSymlink) String() string
type TestWalkFunc ¶
TestWalkFunc is used by TestWalkFiles to traverse the dir. When an error is returned, traversal stops and the surrounding test is marked as failed.