filesystem

package
v0.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 10, 2025 License: Apache-2.0 Imports: 19 Imported by: 5

Documentation

Overview

Package filesystem provides the interface for inventory extraction plugins.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNotRelativeToScanRoots is returned when one of the file or directory to be retrieved or
	// skipped is not relative to any of the scan roots.
	ErrNotRelativeToScanRoots = errors.New("path not relative to any of the scan roots")
	// ErrFailedToOpenFile is returned when opening a file fails.
	ErrFailedToOpenFile = errors.New("failed to open file")
)
View Source
var (
	// ErrExtractorMemoryLimitExceeded is returned when an extractor skips a file
	// due to the extraction process exceeding a configured memory limit.
	ErrExtractorMemoryLimitExceeded = errors.New("extraction failed due to extractor exceeding the configured memory limit")
)

Functions

func ExtractorErrorToFileExtractedResult added in v0.1.2

func ExtractorErrorToFileExtractedResult(err error) stats.FileExtractedResult

ExtractorErrorToFileExtractedResult converts an error returned by an extractor to a FileExtractedResult for stats collection. Converting the error to a result minimizes the memory used for reporting stats.

func InitWalkContext added in v0.1.2

func InitWalkContext(ctx context.Context, config *Config, absScanRoots []*scalibrfs.ScanRoot) (*walkContext, error)

InitWalkContext initializes the walk context for a filesystem walk. It strips all the paths that are expected to be relative to the scan root. This function is exported for TESTS ONLY.

func IsInterestingExecutable added in v0.1.6

func IsInterestingExecutable(api FileAPI) bool

IsInterestingExecutable returns true if the specified file is an executable which may need scanning.

func Run

func Run(ctx context.Context, config *Config) (inventory.Inventory, []*plugin.Status, error)

Run runs the specified extractors and returns their extraction results, as well as info about whether the plugin runs completed successfully.

func RunFS

func RunFS(ctx context.Context, config *Config, wc *walkContext) (inventory.Inventory, []*plugin.Status, error)

RunFS runs the specified extractors and returns their extraction results, as well as info about whether the plugin runs completed successfully. scanRoot is the location of fsys. This method is for testing, use Run() to avoid confusion with scanRoot vs fsys.

Types

type Config

type Config struct {
	Extractors []Extractor
	ScanRoots  []*scalibrfs.ScanRoot
	// Optional: Individual files to extract inventory from. If specified, the
	// extractors will only look at these files during the filesystem traversal.
	// Note that these are not relative to the ScanRoots and thus need to be
	// sub-directories of one of the ScanRoots.
	PathsToExtract []string
	// Optional: If true, only the files in the top-level directories in PathsToExtract are
	// extracted and sub-directories are ignored.
	IgnoreSubDirs bool
	// Optional: Directories that the file system walk should ignore.
	// Note that these are not relative to the ScanRoots and thus need to be
	// sub-directories of one of the ScanRoots.
	// TODO(b/279413691): Also skip local paths, e.g. "Skip all .git dirs"
	DirsToSkip []string
	// Optional: If the regex matches a directory, it will be skipped.
	SkipDirRegex *regexp.Regexp
	// Optional: If the regex matches a glob, it will be skipped.
	SkipDirGlob glob.Glob
	// Optional: Skip files declared in .gitignore files in source repos.
	UseGitignore bool
	// Optional: stats allows to enter a metric hook. If left nil, no metrics will be recorded.
	Stats stats.Collector
	// Optional: Whether to read symlinks.
	ReadSymlinks bool
	// Optional: Limit for visited inodes. If 0, no limit is applied.
	MaxInodes int
	// Optional: Files larger than this size in bytes are skipped. If 0, no limit is applied.
	MaxFileSize int
	// Optional: By default, inventories stores a path relative to the scan root. If StoreAbsolutePath
	// is set, the absolute path is stored instead.
	StoreAbsolutePath bool
	// Optional: If true, print a detailed analysis of the duration of each extractor.
	PrintDurationAnalysis bool
	// Optional: If true, fail the scan if any permission errors are encountered.
	ErrorOnFSErrors bool
}

Config stores the config settings for an extraction run.

type Extractor

type Extractor interface {
	extractor.Extractor
	// FileRequired should return true if the file described by path and file info is
	// relevant for the extractor.
	// Note that the plugin doesn't traverse the filesystem itself but relies on the core
	// library for that.
	FileRequired(api FileAPI) bool
	// Extract extracts inventory data relevant for the extractor from a given file.
	Extract(ctx context.Context, input *ScanInput) (inventory.Inventory, error)
}

Extractor is the filesystem-based inventory extraction plugin, used to extract inventory data from the filesystem such as OS and language packages.

type FileAPI added in v0.1.5

type FileAPI interface {
	// Stat returns the file info for the file.
	Stat() (fs.FileInfo, error)
	Path() string
}

FileAPI is the interface for accessing file information and path.

type ScanInput

type ScanInput struct {
	// FS for file access. This is rooted at Root.
	FS scalibrfs.FS
	// The path of the file to extract, relative to Root.
	Path string
	// The root directory where the extraction file walking started from.
	Root string
	Info fs.FileInfo
	// A reader for accessing contents of the file.
	// Note that the file is closed by the core library, not the plugin.
	Reader io.Reader
}

ScanInput describes one file to extract from.

func (*ScanInput) GetRealPath added in v0.1.5

func (i *ScanInput) GetRealPath() (string, error)

GetRealPath returns the real absolute path of the file on the scanning host's filesystem. If the file is on a virtual filesystem (e.g. a remote container), it is first copied into a temporary directory on the scanning host's filesystem. It's up to the caller to delete the directory once they're done using it.

Directories

Path Synopsis
containers
containerd
Package containerd extracts container package from containerd metadb database.
Package containerd extracts container package from containerd metadb database.
podman
Package podman extracts container inventory from podman database.
Package podman extracts container inventory from podman database.
ffa
unknownbinariesextr
Package unknownbinariesextr identifies binary files on the filesystem and adds them as packages.
Package unknownbinariesextr identifies binary files on the filesystem and adds them as packages.
Package internal contains miscellaneous functions and objects useful within Scalibr
Package internal contains miscellaneous functions and objects useful within Scalibr
units
Package units provides constants for common units.
Package units provides constants for common units.
language
cpp/conanlock
Package conanlock extracts conan.lock files.
Package conanlock extracts conan.lock files.
dart/pubspec
Package pubspec extracts Dart pubspec.lock files.
Package pubspec extracts Dart pubspec.lock files.
dotnet/depsjson
Package depsjson extracts packages from .NET deps.json files.
Package depsjson extracts packages from .NET deps.json files.
dotnet/dotnetpe
Package dotnetpe extracts packages from .NET PE files.
Package dotnetpe extracts packages from .NET PE files.
dotnet/packagesconfig
Package packagesconfig extracts packages from .NET packages.config files.
Package packagesconfig extracts packages from .NET packages.config files.
dotnet/packageslockjson
Package packageslockjson extracts packages.lock.json files.
Package packageslockjson extracts packages.lock.json files.
elixir/mixlock
Package mixlock extracts elixir mix.lock files.
Package mixlock extracts elixir mix.lock files.
erlang/mixlock
Package mixlock extracts erlang mix.lock files.
Package mixlock extracts erlang mix.lock files.
erlang/mixlock/mixlockutils
Package mixlockutils provides common functions for parsing Mix.lock lockfiles.
Package mixlockutils provides common functions for parsing Mix.lock lockfiles.
erlang/mixlock/purl
Package purl converts mixlock package details into a mixlock PackageURL.
Package purl converts mixlock package details into a mixlock PackageURL.
golang/gobinary
Package gobinary extracts packages from buildinfo inside go binaries files.
Package gobinary extracts packages from buildinfo inside go binaries files.
golang/gomod
Package gomod extracts go.mod files.
Package gomod extracts go.mod files.
golang/purl
Package purl converts Go package details into a Go PackageURL.
Package purl converts Go package details into a Go PackageURL.
haskell/cabal
Package cabal extracts cabal.project.freeze files from haskell projects.
Package cabal extracts cabal.project.freeze files from haskell projects.
haskell/stacklock
Package stacklock extracts stack.yaml.lock files from haskell projects.
Package stacklock extracts stack.yaml.lock files from haskell projects.
java/archive
Package archive extracts Java archive files.
Package archive extracts Java archive files.
java/archive/metadata
Package metadata defines a Metadata struct for Java archives.
Package metadata defines a Metadata struct for Java archives.
java/gradlelockfile
Package gradlelockfile extracts pom.xml files.
Package gradlelockfile extracts pom.xml files.
java/gradleverificationmetadataxml
Package gradleverificationmetadataxml extracts Gradle files.
Package gradleverificationmetadataxml extracts Gradle files.
java/groupid
Package groupid provides functionality for retrieving the group ID of a Java package.
Package groupid provides functionality for retrieving the group ID of a Java package.
java/javalockfile
Package javalockfile provides shared structures for Java extractors.
Package javalockfile provides shared structures for Java extractors.
java/pomxml
Package pomxml extracts pom.xml files.
Package pomxml extracts pom.xml files.
java/pomxmlnet
Package pomxmlnet extracts Maven's pom.xml format with transitive dependency resolution.
Package pomxmlnet extracts Maven's pom.xml format with transitive dependency resolution.
java/purl
Package purl converts a package to a Maven type PackageURL.
Package purl converts a package to a Maven type PackageURL.
javascript/bunlock
Package bunlock extracts bun.lock files
Package bunlock extracts bun.lock files
javascript/internal/commitextractor
Package commitextractor provides a function to extract commit hash from the full git URL
Package commitextractor provides a function to extract commit hash from the full git URL
javascript/packagejson
Package packagejson extracts package.json files.
Package packagejson extracts package.json files.
javascript/packagelockjson
Package packagelockjson extracts package-lock.json files.
Package packagelockjson extracts package-lock.json files.
javascript/pnpmlock
Package pnpmlock extracts pnpm-lock.yaml files.
Package pnpmlock extracts pnpm-lock.yaml files.
javascript/purl
Package purl converts NPM package details into an NPM PackageURL.
Package purl converts NPM package details into an NPM PackageURL.
javascript/yarnlock
Package yarnlock extracts NPC yarn.lock files.
Package yarnlock extracts NPC yarn.lock files.
php/composerlock
Package composerlock extracts composer.lock files.
Package composerlock extracts composer.lock files.
python/condameta
Package condameta extracts Conda package metadata from conda-meta JSON files.
Package condameta extracts Conda package metadata from conda-meta JSON files.
python/pdmlock
Package pdmlock extracts pdm.lock files.
Package pdmlock extracts pdm.lock files.
python/pipfilelock
Package pipfilelock extracts Pipfile.lock files.
Package pipfilelock extracts Pipfile.lock files.
python/poetrylock
Package poetrylock extracts poetry.lock files.
Package poetrylock extracts poetry.lock files.
python/pypipurl
Package pypipurl converts a package to a PyPI type PackageURL.
Package pypipurl converts a package to a PyPI type PackageURL.
python/requirements
Package requirements extracts requirements files.
Package requirements extracts requirements files.
python/requirementsnet
Package requirementsnet extracts requirements files with .
Package requirementsnet extracts requirements files with .
python/setup
Package setup extracts packages from setup.py.
Package setup extracts packages from setup.py.
python/uvlock
Package uvlock extracts uv.lock files.
Package uvlock extracts uv.lock files.
python/wheelegg
Package wheelegg extracts wheel and egg files.
Package wheelegg extracts wheel and egg files.
r/renvlock
Package renvlock extracts renv.lock files.
Package renvlock extracts renv.lock files.
ruby/gemfilelock
Package gemfilelock extracts Gemfile.lock files.
Package gemfilelock extracts Gemfile.lock files.
ruby/gemspec
Package gemspec extracts *.gemspec files.
Package gemspec extracts *.gemspec files.
rust/cargoauditable
Package cargoauditable extracts dependencies from cargo auditable inside rust binaries.
Package cargoauditable extracts dependencies from cargo auditable inside rust binaries.
rust/cargolock
Package cargolock extracts Cargo.lock files for rust projects
Package cargolock extracts Cargo.lock files for rust projects
rust/cargotoml
Package cargotoml extracts Cargo.toml files for rust projects
Package cargotoml extracts Cargo.toml files for rust projects
swift/packageresolved
Package packageresolved extracts Package.resolved files
Package packageresolved extracts Package.resolved files
swift/podfilelock
Package podfilelock extracts dependencies from Podfile.lock files.
Package podfilelock extracts dependencies from Podfile.lock files.
swift/swiftutils
Package swiftutils provides utilities for parsing Swift podfiles.
Package swiftutils provides utilities for parsing Swift podfiles.
Package list provides a public list of SCALIBR-internal extraction plugins.
Package list provides a public list of SCALIBR-internal extraction plugins.
misc
chrome/extensions
Package extensions extracts chrome extensions.
Package extensions extracts chrome extensions.
vscodeextensions
Package vscodeextensions extracts vscode extensions.
Package vscodeextensions extracts vscode extensions.
wordpress/plugins
Package plugins extracts packages from installed Wordpress plugins.
Package plugins extracts packages from installed Wordpress plugins.
os
apk
Package apk extracts packages from the APK database.
Package apk extracts packages from the APK database.
apk/apkutil
Package apkutil provides utilities for reading APK package records.
Package apkutil provides utilities for reading APK package records.
apk/metadata
Package metadata defines a Metadata struct for apk packages.
Package metadata defines a Metadata struct for apk packages.
cos
Package cos extracts OS packages from Container Optimized OSes (go/cos).
Package cos extracts OS packages from Container Optimized OSes (go/cos).
cos/metadata
Package metadata defines a metadata struct for COS packages.
Package metadata defines a metadata struct for COS packages.
dpkg
Package dpkg extracts packages from dpkg database.
Package dpkg extracts packages from dpkg database.
dpkg/metadata
Package metadata defined a Metadata struct for DPKG packages.
Package metadata defined a Metadata struct for DPKG packages.
ecosystem
Package ecosystem converts OS package details into PackageURLs.
Package ecosystem converts OS package details into PackageURLs.
flatpak
Package flatpak extracts packages from flatpak metainfo files.
Package flatpak extracts packages from flatpak metainfo files.
flatpak/metadata
Package metadata defines a Metadata struct for flatpak packages.
Package metadata defines a Metadata struct for flatpak packages.
homebrew
Package homebrew extracts package information from OSX homebrew INSTALL_RECEIPT.json files.
Package homebrew extracts package information from OSX homebrew INSTALL_RECEIPT.json files.
kernel/module
Package module extracts .ko files from kernel modules.
Package module extracts .ko files from kernel modules.
kernel/module/metadata
Package metadata defines a metadata struct for kernel modules.
Package metadata defines a metadata struct for kernel modules.
kernel/vmlinuz
Package vmlinuz extracts information about vmlinuz compressed kernel images.
Package vmlinuz extracts information about vmlinuz compressed kernel images.
kernel/vmlinuz/metadata
Package metadata defines a metadata struct for kernel vmlinuz files.
Package metadata defines a metadata struct for kernel vmlinuz files.
macapps
Package macapps extracts applications data from Info.plist files of OS X devices.
Package macapps extracts applications data from Info.plist files of OS X devices.
nix
Package nix extracts packages from the Nix store directory.
Package nix extracts packages from the Nix store directory.
nix/metadata
Package metadata defines a metadata struct for nix packages.
Package metadata defines a metadata struct for nix packages.
osrelease
Package osrelease parses the os-release file.
Package osrelease parses the os-release file.
pacman
Package pacman extracts packages from archlinux desc file.
Package pacman extracts packages from archlinux desc file.
pacman/metadata
Package metadata defines a metadata struct for arch packages.
Package metadata defines a metadata struct for arch packages.
portage
Package portage extracts packages from portage database.
Package portage extracts packages from portage database.
portage/metadata
Package metadata defines a metadata struct for portage packages.
Package metadata defines a metadata struct for portage packages.
purl
Package purl converts OS package details into PackageURLs.
Package purl converts OS package details into PackageURLs.
rpm
Package rpm extracts packages from rpm database.
Package rpm extracts packages from rpm database.
rpm/metadata
Package metadata defines a metadata struct for rpm packages.
Package metadata defines a metadata struct for rpm packages.
snap
Package snap extracts snap packages
Package snap extracts snap packages
snap/metadata
Package metadata defines a metadata struct for SNAP packages.
Package metadata defines a metadata struct for SNAP packages.
Package osv defines OSV-specific fields for parsed source packages.
Package osv defines OSV-specific fields for parsed source packages.
sbom
cdx
Package cdx extracts software dependencies from an CycloneDX SBOM.
Package cdx extracts software dependencies from an CycloneDX SBOM.
cdx/metadata
Package metadata defines a Metadata struct for CDX packages.
Package metadata defines a Metadata struct for CDX packages.
cdx/purl
Package purl extracts the PURL from a CDX package.
Package purl extracts the PURL from a CDX package.
spdx
Package spdx extracts software dependencies from an SPDX SBOM.
Package spdx extracts software dependencies from an SPDX SBOM.
spdx/metadata
Package metadata defines a Metadata struct for SPDX packages.
Package metadata defines a Metadata struct for SPDX packages.
spdx/purl
Package purl extracts the PURL from an SPDX package.
Package purl extracts the PURL from an SPDX package.
Package secrets contains a Scalibr filesystem Extractor that wraps the Veles secret scanning library to find secrets (i.e.
Package secrets contains a Scalibr filesystem Extractor that wraps the Veles secret scanning library to find secrets (i.e.
Package simplefileapi provides a fake implementation of the filesystem.FileAPI interface.
Package simplefileapi provides a fake implementation of the filesystem.FileAPI interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL