pachd

package
v2.12.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 15, 2025 License: Apache-2.0 Imports: 114 Imported by: 0

Documentation

Overview

Package pachd implements the Pachyderm dæmon and its various modes.

Callers need only provide a context and a configuration.

Adding a new mode

The idea is that there is a builder for each mode; each builder is responsible for building and running an instance of the single type daemon representing a pachd instance.

To add a new mode one will at least add a new builder; one may also need to add new members to daemon. Daemon should contain only those members needed at run time for any mode; other, transient, values should be members of the pertinent builder.

Package pachd needs to be documented.

TODO: document

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AdminEnv added in v2.8.0 

func AdminEnv(senv serviceenv.ServiceEnv, paused bool) admin_server.Env

func AuthEnv added in v2.8.0 

func AuthEnv(senv serviceenv.ServiceEnv, txnEnv *txnenv.TransactionEnv) auth_server.Env

func DebugEnv added in v2.8.0 

func DebugEnv(env serviceenv.ServiceEnv) debug_server.Env

func EnterpriseMode 

func EnterpriseMode(ctx context.Context, config *pachconfig.EnterpriseServerConfiguration) error

EnterpriseMode runs an enterprise-mode pachd.

Enterprise mode is the enterprise server which is used to manage multiple Pachyderm installations.

func FullMode 

func FullMode(ctx context.Context, config *pachconfig.PachdFullConfiguration) error

FullMode runs a full-mode pachd.

Full mode is that standard pachd which users interact with using pachctl and which manages pipelines, files and so forth.

func NewTestPachd added in v2.8.0 

func NewTestPachd(t testing.TB, opts ...TestPachdOption) *client.APIClient

NewTestPachd creates an environment suitable for non-k8s tests and then calls pachd.NewFull with that environment.

func PFSEnv added in v2.8.0 

func PFSEnv(env serviceenv.ServiceEnv, txnEnv *txnenv.TransactionEnv) (*pfs_server.Env, error)

func PFSWorkerEnv added in v2.8.0 

func PFSWorkerEnv(env serviceenv.ServiceEnv) (*pfs_server.WorkerEnv, error)

func PPSEnv added in v2.8.0 

func PPSEnv(senv serviceenv.ServiceEnv, txnEnv *txnenv.TransactionEnv, reporter *metrics.Reporter) pps_server.Env

func PachwEnv added in v2.8.0 

func PachwEnv(env serviceenv.ServiceEnv) (*pachw_server.Env, error)

func PachwMode 

func PachwMode(ctx context.Context, config *pachconfig.PachdFullConfiguration) error

PachwMode runs a pachw-mode pachd. When in pachw mode, the pachd instance processes storage and url tasks.

func PausedMode 

func PausedMode(ctx context.Context, config *pachconfig.PachdFullConfiguration) error

PausedMode runs a paused-mode pachd.

Paused mode is a restricted mode which runs Pachyderm read-only in order to take offline backups.

func SidecarMode 

func SidecarMode(ctx context.Context, config *pachconfig.PachdFullConfiguration) error

SidecarMode runs a sidecar-mode pachd.

Sidecar mode is run as a sidecar in a pipeline pod; it provides services to the pipeline worker code running in that pod.

func SnapshotEnv added in v2.12.0 

func SnapshotEnv(env serviceenv.ServiceEnv) (*fileset.Storage, error)

func StorageEnv added in v2.10.0 

func StorageEnv(env serviceenv.ServiceEnv) (*storage.Env, error)

Types

type Env added in v2.8.0 

type Env struct {
	DB               *pachsql.DB
	DirectDB         *pachsql.DB
	DBListenerConfig string
	Bucket           *obj.Bucket
	EtcdClient       *clientv3.Client
	Listener         net.Listener
	K8sObjects       []runtime.Object
	GetLokiClient    func() (*lokiclient.Client, error)
}

type Full added in v2.8.0 

type Full struct {
	// contains filtered or unexported fields
}

func BuildTestPachd added in v2.10.0 

func BuildTestPachd(ctx context.Context, opts ...TestPachdOption) (*Full, *cleanup.Cleaner, error)

BuildTestPachd returns a test pachd that can be run outside of tests. The returned cleanup handler frees all ephemeral resources associated with the instance.

func NewFull added in v2.8.0 

func NewFull(env Env, config pachconfig.PachdFullConfiguration, opt *FullOption) *Full

NewFull sets up a new Full pachd and returns it.

func (*Full) AwaitAuth added in v2.10.0 

func (pd *Full) AwaitAuth(ctx context.Context)

AwaitAuth returns when auth is ready. It must be called after Run() starts.

func (*Full) PachClient added in v2.10.0 

func (pd *Full) PachClient(ctx context.Context) (*client.APIClient, error)

PachClient returns a pach client that can talk to the server with root privileges.

func (*Full) Run added in v2.8.0 

func (b *Full) Run(ctx context.Context) error

Run runs all the setup steps serially, returning the first error encountered. Then all the background tasks are run concurrently, if any of them error then the others are cancelled, and Run returns the error.

func (*Full) Snapshotter added in v2.12.0 

func (pd *Full) Snapshotter() *snapshot_server.Snapshotter

Snapshotter is meant for use in tests which need to get a snapshot.Snapshotter.

type FullOption added in v2.10.0 

type FullOption struct {
	DesiredState *migrations.State
}

type PreFlight added in v2.8.0 

type PreFlight struct {
	// contains filtered or unexported fields
}

PreFlight is a minimal pachd for running preflight checks.

func NewPreflight added in v2.8.0 

func NewPreflight(env PreFlightEnv, config pachconfig.PachdPreflightConfiguration) *PreFlight

func (*PreFlight) Run added in v2.8.0 

func (b *PreFlight) Run(ctx context.Context) error

Run runs all the setup steps serially, returning the first error encountered. Then all the background tasks are run concurrently, if any of them error then the others are cancelled, and Run returns the error.

type PreFlightEnv added in v2.8.0 

type PreFlightEnv struct {
	// DB should be a direct database connection.
	DB *sqlx.DB
}

type RestoreSnapshot added in v2.12.0 

type RestoreSnapshot struct {
	// contains filtered or unexported fields
}

RestoreSnapshot is a minimal pachd for restoring from a snapshot.

func NewRestoreSnapshot added in v2.12.0 

func NewRestoreSnapshot(env RestoreSnapshotEnv, config pachconfig.PachdRestoreSnapshotConfiguration) *RestoreSnapshot

func (*RestoreSnapshot) Run added in v2.12.0 

func (b *RestoreSnapshot) Run(ctx context.Context) error

Run runs all the setup steps serially, returning the first error encountered. Then all the background tasks are run concurrently, if any of them error then the others are cancelled, and Run returns the error.

type RestoreSnapshotEnv added in v2.12.0 

type RestoreSnapshotEnv struct {
	// DB should be a direct database connection.
	DB *sqlx.DB
}

type TestPachdOption added in v2.10.0 

type TestPachdOption struct {
	MutateContext    func(ctx context.Context) context.Context
	MutateEnv        func(env *Env)
	MutateConfig     func(config *pachconfig.PachdFullConfiguration)
	MutatePachd      func(full *Full)
	MutateFullOption func(fullOption *FullOption)
	OnReady          func(ctx context.Context, full *Full) error
	Cleanup          func(ctx context.Context) error
	CopyDBConfig     *dockertestenv.DBConfig // will copy the database configuration
	// contains filtered or unexported fields
}

TestPachdOption is used to customize a testpachd.

func ActivateAuthOption added in v2.10.0 

func ActivateAuthOption(rootToken string) TestPachdOption

ActivateAuthOption is an option that activates auth inside the created pachd. Outside of tests, you must manually call pachd.AwaitAuth(ctx).

func GetK8sClient added in v2.12.0 

func GetK8sClient(k *kubernetes.Interface) TestPachdOption

GetK8sClient is an option that sets k to the internal fake k8s client. Mutations made to this client will be visible to Pachyderm.

func NoLogToFileOption added in v2.10.0 

func NoLogToFileOption() TestPachdOption

NoLogToFileOption is an option that disables NewTestPachd's default behavior of logging pachd logs to a file.

func PJSWorkerAuthOption added in v2.12.0 

func PJSWorkerAuthOption(pjsWorkerAuthToken string) TestPachdOption

func WithS3Server added in v2.12.0 

func WithS3Server(t *testing.T, addr *string) TestPachdOption

WithS3Server runs the S3 gateway and sets the provided string pointer to its address.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.