squashfs

package module
v1.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 18, 2025 License: MIT Imports: 15 Imported by: 3

README

GoDoc

squashfs

This is a read-only implementation of squashfs initially meant to be use with go-fuse.

Since then, golang added io/fs and fuse support was moved to a fuse tag, which means this module can be either used with go-fuse, or as a simple io/fs-compliant squashfs file reader.

Tags

The following tags can be specified on build to enable/disable features:

  • fuse adds methods to the Inode object to interact with fuse
  • xz adds a dependency on github.com/ulikunitz/xz to support XZ compressed files
  • zstd adds a dependency on github.com/klauspost/compress/zstd to support ZSTD compressed files

Note: By default, only GZip compression is supported. Other compression formats mentioned in the SquashFS specification (LZMA, LZO, LZ4) are not currently implemented via build tags, but can be added by manually registering decompressors.

Example usage

Basic file access

sqfs, err := squashfs.Open("file.squashfs")
if err != nil {
	return err
}
defer sqfs.Close()

// sqfs can be used as a regular fs.FS
data, err := fs.ReadFile(sqfs, "dir/file.txt")
if err != nil {
	return err
}

// Or serve files over HTTP
http.Handle("/", http.FileServer(sqfs))

Reading directories

// List directory contents
entries, err := sqfs.ReadDir("some/directory")
if err != nil {
	return err
}

// Process directory entries
for _, entry := range entries {
	fmt.Printf("Name: %s, IsDir: %v\n", entry.Name(), entry.IsDir())
	
	// Get more info if needed
	info, err := entry.Info()
	if err != nil {
		return err
	}
	fmt.Printf("Size: %d, Mode: %s\n", info.Size(), info.Mode())
}

// Read a symlink target
target, err := sqfs.Readlink("path/to/symlink")
if err != nil {
	return err
}
fmt.Printf("Symlink points to: %s\n", target)

Custom compression support

// Register XZ support (requires "xz" build tag)
import (
	"github.com/KarpelesLab/squashfs"
	"github.com/ulikunitz/xz"
)

// Register XZ decompressor at init time
func init() {
	squashfs.RegisterDecompressor(squashfs.XZ, squashfs.MakeDecompressorErr(xz.NewReader))
}

For more examples, see the test files in the project.

File format

Some documentation is available online on SquashFS.

Features

  • Read-only implementation of squashfs compatible with Go's io/fs interface
  • Optional FUSE support with the fuse build tag
  • Support for GZip compression by default, with XZ and ZSTD available via build tags
  • Extensible compression support through the RegisterDecompressor API
  • Directory index support for fast access to files in large directories
  • Symlink support
  • CLI tool for exploring and extracting files from SquashFS archives

CLI Tool

A command-line interface tool is provided in the cmd/sqfs directory. This tool allows you to:

  • List files in a SquashFS archive
  • View the contents of files inside a SquashFS archive
  • Display detailed information about SquashFS archives

Usage

sqfs - SquashFS CLI tool

Usage:
  sqfs ls <squashfs_file> [<path>]          List files in SquashFS (optionally in a specific path)
  sqfs cat <squashfs_file> <file>           Display contents of a file in SquashFS
  sqfs info <squashfs_file>                 Display information about a SquashFS archive
  sqfs help                                 Show this help message

Examples:
  sqfs ls archive.squashfs                  List all files at the root of archive.squashfs
  sqfs ls archive.squashfs lib              List all files in the lib directory
  sqfs cat archive.squashfs dir/file.txt    Display contents of file.txt from archive.squashfs
  sqfs info archive.squashfs                Show metadata about the SquashFS archive

Installing the CLI Tool

go install github.com/KarpelesLab/squashfs/cmd/sqfs@latest

To install with additional compression support:

go install -tags "xz zstd" github.com/KarpelesLab/squashfs/cmd/sqfs@latest

Performance

As of November 2024, directory indexes are now used for efficient file lookup in large directories, significantly improving performance for random file access.

Documentation

Index

Constants

View Source 
const SuperblockSize = 96

Variables

View Source 
var (
	// ErrInvalidFile is returned when the file format is not recognized as SquashFS
	ErrInvalidFile = errors.New("invalid file, squashfs signature not found")

	// ErrInvalidSuper is returned when the superblock data is corrupted or invalid
	ErrInvalidSuper = errors.New("invalid squashfs superblock")

	// ErrInvalidVersion is returned when the SquashFS version is not 4.0
	// This library only supports SquashFS 4.0 format
	ErrInvalidVersion = errors.New("invalid file version, expected squashfs 4.0")

	// ErrInodeNotExported is returned when trying to access an inode that isn't in the export table
	ErrInodeNotExported = errors.New("unknown squashfs inode and no NFS export table")

	// ErrNotDirectory is returned when attempting to perform directory operations on a non-directory
	ErrNotDirectory = errors.New("not a directory")

	// ErrTooManySymlinks is returned when symlink resolution exceeds the maximum depth
	// This prevents infinite loops in symlink resolution
	ErrTooManySymlinks = errors.New("too many levels of symbolic links")
)

Package-specific error variables that can be used with errors.Is() for error handling.

Functions

func RegisterDecompressor 

func RegisterDecompressor(method Compression, dcomp Decompressor)

RegisterDecompressor can be used to register a decompressor for squashfs. By default GZip is supported. The method shall take a buffer and return a decompressed buffer.

Types

type Compression added in v0.1.4 

type Compression uint16

Compression represents the compression algorithm used in a SquashFS filesystem. Different compression methods can be used to optimize for size or decompression speed. By default, only GZip compression is supported. Additional compression formats can be enabled through build tags or manually registering decompressors. 

const (
	GZip Compression = iota + 1 // GZip compression (zlib, always supported)
	LZMA                        // LZMA compression (currently not implemented via build tag)
	LZO                         // LZO compression (currently not implemented via build tag)
	XZ                          // XZ compression (enabled with "xz" build tag)
	LZ4                         // LZ4 compression (currently not implemented via build tag)
	ZSTD                        // Zstandard compression (enabled with "zstd" build tag)
)

func (Compression) String added in v0.1.4 

func (s Compression) String() string

type Decompressor 

type Decompressor func(buf []byte) ([]byte, error)

func MakeDecompressor 

func MakeDecompressor(dec func(r io.Reader) io.ReadCloser) Decompressor

MakeDecompressor allows using a decompressor made for archive/zip with SquashFs. It has some overhead as instead of simply dealing with buffer this uses the reader/writer API, but should allow to easily handle some formats.

Example use: * squashfs.RegisterDecompressor(squashfs.ZSTD, squashfs.MakeDecompressor(zstd.ZipDecompressor())) * squashfs.RegisterDecompressor(squashfs.LZ4, squashfs.MakeDecompressor(lz4.NewReader)))

func MakeDecompressorErr added in v0.1.4 

func MakeDecompressorErr(dec func(r io.Reader) (io.ReadCloser, error)) Decompressor

MakeDecompressorErr is similar to MakeDecompressor but the factory method also returns an error.

Example use: * squashfs.RegisterDecompressor(squashfs.LZMA, squashfs.MakeDecompressorErr(lzma.NewReader)) * squashfs.RegisterDecompressor(squashfs.XZ, squashfs.MakeDecompressorErr(xz.NewReader))

type DirIndexEntry added in v1.1.0 

type DirIndexEntry struct {
	Index uint32 // position in the directory data
	Start uint32 // block offset
	Name  string // filename at this index point
}

DirIndexEntry represents an entry in a directory index. Directory indices allow fast lookups in large directories by providing a sorted index of filenames and their positions in the directory data.

type File 

type File struct {
	*io.SectionReader
	// contains filtered or unexported fields
}

File is a convience object allowing using an inode as if it was a regular file

func (*File) Close 

func (f *File) Close() error

Close actually does nothing and exists to comply with fs.File

func (*File) Stat 

func (f *File) Stat() (fs.FileInfo, error)

Stat returns the details of the open file

func (*File) Sys added in v0.1.5 

func (f *File) Sys() any

Sys returns a *Inode object for this file

type FileDir 

type FileDir struct {
	// contains filtered or unexported fields
}

FileDir is a convenience object allowing using a dir inode as if it was a regular file

func (*FileDir) Close 

func (d *FileDir) Close() error

Close resets the dir reader

func (*FileDir) Read 

func (d *FileDir) Read(p []byte) (int, error)

Read on a directory is invalid and will always fail

func (*FileDir) ReadDir 

func (d *FileDir) ReadDir(n int) ([]fs.DirEntry, error)

func (*FileDir) Stat 

func (d *FileDir) Stat() (fs.FileInfo, error)

Stat returns details on the file

func (*FileDir) Sys added in v0.1.5 

func (d *FileDir) Sys() any

Sys returns a *inode object for this file, similar to calling Stat().Sys()

type Flags added in v0.1.4 

type Flags uint16
const (
	UNCOMPRESSED_INODES Flags = 1 << iota
	UNCOMPRESSED_DATA
	CHECK
	UNCOMPRESSED_FRAGMENTS
	NO_FRAGMENTS
	ALWAYS_FRAGMENTS
	DUPLICATES
	EXPORTABLE
	UNCOMPRESSED_XATTRS
	NO_XATTRS
	COMPRESSOR_OPTIONS
	UNCOMPRESSED_IDS
)

func (Flags) Has added in v0.1.4 

func (f Flags) Has(what Flags) bool

func (Flags) String added in v0.1.4 

func (f Flags) String() string

type Inode 

type Inode struct {
	Type    Type   // file type (regular file, directory, symlink, etc.)
	Perm    uint16 // permission bits
	UidIdx  uint16 // user ID index (in the ID table)
	GidIdx  uint16 // group ID index (in the ID table)
	ModTime int32  // modification time (Unix timestamp)
	Ino     uint32 // inode number (unique identifier)

	StartBlock uint64 // start of data blocks
	NLink      uint32 // number of hard links
	Size       uint64 // file size in bytes (interpretation varies by type)
	Offset     uint32 // offset within block (uint16 for directories)
	ParentIno  uint32 // parent directory's inode number (for directories)
	SymTarget  []byte // target path for symlinks
	IdxCount   uint16 // count of directory index entries (for extended directories)
	XattrIdx   uint32 // extended attribute index if present
	Sparse     uint64 // sparse file information

	// fragment information for file data that doesn't fill a complete block
	FragBlock uint32 // fragment block index
	FragOfft  uint32 // offset within fragment block

	// data block information
	Blocks     []uint32         // block sizes, possibly with compression flags
	BlocksOfft []uint64         // offsets for each block from StartBlock
	DirIndex   []*DirIndexEntry // directory index entries for fast lookups
	// contains filtered or unexported fields
}

Inode represents a file, directory, or other filesystem object in a SquashFS filesystem. It contains all the metadata and references to data blocks that make up the file's contents. Inodes implement various interfaces like io.ReaderAt to provide file-like access to their contents.

func (*Inode) AddRef 

func (i *Inode) AddRef(count uint64) uint64

AddRef atomatically increments the inode's refcount and returns the new value. This is mainly useful when using fuse and can be safely ignored.

func (*Inode) DelRef 

func (i *Inode) DelRef(count uint64) uint64

DelRef atomatically decrements the inode's refcount and returns the new value. This is mainly useful when using fuse and can be safely ignored.

func (*Inode) GetGid added in v0.1.5 

func (i *Inode) GetGid() uint32

GetGid returns inode's group id, or zero if an error happens

func (*Inode) GetUid added in v0.1.5 

func (i *Inode) GetUid() uint32

GetUid returns inode's owner uid, or zero if an error happens

func (*Inode) IsDir 

func (i *Inode) IsDir() bool

IsDir returns true if the inode is a directory inode.

func (*Inode) Mode 

func (i *Inode) Mode() fs.FileMode

Mode returns the inode's mode as fs.FileMode

func (*Inode) OpenFile 

func (ino *Inode) OpenFile(name string) fs.File

OpenFile returns a fs.File for a given inode. If the file is a directory, the returned object will implement fs.ReadDirFile. If it is a regular file it will also implement io.Seeker.

func (*Inode) ReadAt 

func (i *Inode) ReadAt(p []byte, off int64) (int, error)
func (i *Inode) Readlink() ([]byte, error)

Readlink returns the inode's link

type Option added in v0.1.5 

type Option func(sb *Superblock) error

func InodeOffset added in v0.1.5 

func InodeOffset(inoOfft uint64) Option

type Superblock 

type Superblock struct {
	Magic             uint32 // magic identifier
	InodeCnt          uint32 // number of inodes in filesystem
	ModTime           int32  // creation unix time as int32 (will stop working in 2038)
	BlockSize         uint32 // size of a single data block, must match 1<<BlockLog
	FragCount         uint32
	Comp              Compression // Compression used, usually GZip
	BlockLog          uint16
	Flags             Flags // squashfs flags
	IdCount           uint16
	VMajor            uint16
	VMinor            uint16
	RootInode         inodeRef // inode number/reference of root
	BytesUsed         uint64
	IdTableStart      uint64
	XattrIdTableStart uint64
	InodeTableStart   uint64
	DirTableStart     uint64
	FragTableStart    uint64
	ExportTableStart  uint64
	// contains filtered or unexported fields
}

Superblock is the main object representing a squashfs image, and exposes various information about the file system. It implements Go's standard fs.FS, fs.ReadDirFS, and fs.StatFS interfaces, allowing it to be used as a drop-in replacement for any code that works with the standard filesystem interfaces.

The Superblock provides access to all the internal data structures of the SquashFS image, including inodes, compression information, and directory structures. For most use cases, you can use the standard fs methods (Open, ReadDir, etc.) without needing to understand the underlying implementation.

For advanced use cases, the Superblock also provides direct access to internal structures like inodes and directory entries.

func New 

func New(fs io.ReaderAt, options ...Option) (*Superblock, error)

New returns a new instance of superblock for a given io.ReaderAt that can be used to access files inside squashfs.

func Open added in v0.1.5 

func Open(file string, options ...Option) (*Superblock, error)

Open returns a new instance of superblock for a given file that can be used to access files inside squashfs. The file will be closed by the garbage collector or when Close() is called on the superblock.

func (*Superblock) Close added in v0.1.5 

func (sb *Superblock) Close() error

Close will close the underlying file when a filesystem was open with Open()

func (*Superblock) FindInode added in v0.1.5 

func (s *Superblock) FindInode(name string, followSymlinks bool) (*Inode, error)

FindInode returns the inode for a given path. If followSymlink is false and a symlink is found in the path, it will be followed anyway. If however the target file is a symlink, then its inode will be returned.

func (*Superblock) FindInodeUnder added in v1.0.0 

func (s *Superblock) FindInodeUnder(cur *Inode, name string, followSymlinks bool) (*Inode, error)

FindInodeUnder returns an inode for a path starting at a given different inode

Note that it is not possible to access directories outside the given path, including using symlinks, as this effectively acts as a chroot. This can be useful to implement fs.Sub

func (*Superblock) GetInode 

func (sb *Superblock) GetInode(ino uint64) (*Inode, error)

func (*Superblock) GetInodeRef 

func (sb *Superblock) GetInodeRef(inor inodeRef) (*Inode, error)

func (*Superblock) Lstat added in v0.1.5 

func (sb *Superblock) Lstat(name string) (fs.FileInfo, error)

Lstat will return stats for a given path inside the sqhashfs archive. If the target is a symbolic link, data on the link itself will be returned.

func (*Superblock) Open 

func (sb *Superblock) Open(name string) (fs.File, error)

Open returns a fs.File for a given path, which can be a different object depending if the file is a regular file or a directory.

func (*Superblock) ReadDir 

func (sb *Superblock) ReadDir(name string) ([]fs.DirEntry, error)

ReadDir implements fs.ReadDirFS and allows listing any directory inside the archive 

func (sb *Superblock) Readlink(name string) (string, error)

Readlink allows reading the value of a symbolic link inside the archive.

func (*Superblock) SetInodeOffset 

func (s *Superblock) SetInodeOffset(offt uint64)

SetInodeOffset allows setting the inode offset used for interacting with fuse. This can be safely ignored if not using fuse or when mounting only a single squashfs via fuse.

func (*Superblock) Stat 

func (sb *Superblock) Stat(name string) (fs.FileInfo, error)

Stat will return stats for a given path inside the squashfs archive

func (*Superblock) UnmarshalBinary 

func (s *Superblock) UnmarshalBinary(data []byte) error

UnmarshalBinary reads a binary header values into Superblock

type Type added in v0.1.5 

type Type uint16

Type represents the type of a file or directory in a SquashFS filesystem. SquashFS has both basic and extended types, where extended types provide additional information or capabilities. 

const (
	DirType       Type = iota + 1 // Basic directory
	FileType                      // Basic regular file
	SymlinkType                   // Symbolic link
	BlockDevType                  // Block device
	CharDevType                   // Character device
	FifoType                      // Named pipe (FIFO)
	SocketType                    // UNIX domain socket
	XDirType                      // Extended directory with optional index information
	XFileType                     // Extended file with additional metadata
	XSymlinkType                  // Extended symbolic link
	XBlockDevType                 // Extended block device
	XCharDevType                  // Extended character device
	XFifoType                     // Extended named pipe
	XSocketType                   // Extended UNIX domain socket
)

func (Type) Basic added in v0.1.5 

func (t Type) Basic() Type

Basic returns the type as a basic type (ie. XDirType.Basic() == DirType)

func (Type) IsDir added in v0.1.5 

func (t Type) IsDir() bool
func (t Type) IsSymlink() bool

func (Type) Mode added in v0.1.5 

func (t Type) Mode() fs.FileMode

Mode returns a fs.FileMode for this type that contains no permissions, only the file's type

Source Files

View all Source files

Directories

Path Synopsis
cmd
sqfs Module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.