Documentation
¶
Overview ¶
Package ollama provides a client for interacting with an Ollama registry which pushes and pulls model manifests and layers as defined by the ollama.com/manifest.
Index ¶
- Constants
- Variables
- func CompleteName(name string) string
- func DefaultCache() (*blob.DiskCache, error)
- func UserAgent() string
- func WithTrace(ctx context.Context, t *Trace) context.Context
- type Error
- type Layer
- type Manifest
- type PushParams
- type Registry
- func (r *Registry) Pull(ctx context.Context, name string) error
- func (r *Registry) Push(ctx context.Context, name string, p *PushParams) error
- func (r *Registry) Resolve(ctx context.Context, name string) (*Manifest, error)
- func (r *Registry) ResolveLocal(name string) (*Manifest, error)
- func (r *Registry) Unlink(name string) (ok bool, _ error)
- type Trace
Examples ¶
Constants ¶
const ( // DefaultChunkingThreshold is the threshold at which a layer should be // split up into chunks when downloading. DefaultChunkingThreshold = 64 << 20 )
Defaults
const DefaultMask = "registry.ollama.ai/library/_:latest"
Variables ¶
var ( // ErrModelNotFound is returned when a manifest is not found in the // cache or registry. ErrModelNotFound = errors.New("model not found") // ErrManifestInvalid is returned when a manifest found in a local or // remote cache is invalid. ErrManifestInvalid = errors.New("invalid manifest") // ErrMissingModel is returned when the model part of a name is missing // or invalid. ErrNameInvalid = errors.New("invalid or missing name") // ErrCached is passed to [Trace.PushUpdate] when a layer already // exists. It is a non-fatal error and is never returned by [Registry.Push]. ErrCached = errors.New("cached") // ErrIncomplete is returned by [Registry.Pull] when a model pull was // incomplete due to one or more layer download failures. Users that // want specific errors should use [WithTrace]. ErrIncomplete = errors.New("incomplete") )
Errors
Functions ¶
func CompleteName ¶
CompleteName returns a fully qualified name by merging the given name with the default mask. If the name is already fully qualified, it is returned unchanged.
func DefaultCache ¶
DefaultCache returns the default cache used by the registry. It is configured from the OLLAMA_MODELS environment variable, or defaults to $HOME/.ollama/models, or, if an error occurs obtaining the home directory, it uses the current working directory.
Types ¶
type Error ¶
type Error struct {
Code string `json:"code"`
Message string `json:"message"`
// contains filtered or unexported fields
}
Error is the standard error returned by Ollama APIs. It can represent a single or multiple error response.
Single error responses have the following format:
{"code": "optional_code","error":"error message"}
Multiple error responses have the following format:
{"errors": [{"code": "optional_code","message":"error message"}]}
Note, that the error field is used in single error responses, while the message field is used in multiple error responses.
In both cases, the code field is optional and may be empty.
func (*Error) UnmarshalJSON ¶
UnmarshalJSON implements json.Unmarshaler.
type Layer ¶
type Layer struct {
Digest blob.Digest `json:"digest"`
MediaType string `json:"mediaType"`
Size int64 `json:"size"`
}
Layer is a layer in a model.
type Manifest ¶
type Manifest struct {
Name string `json:"-"` // the canonical name of the model
Data []byte `json:"-"` // the raw data of the manifest
Layers []*Layer `json:"layers"`
// For legacy reasons, we still have to download the config layer.
Config *Layer `json:"config"`
}
Manifest represents a ollama.com/manifest.
func (Manifest) MarshalJSON ¶
MarshalJSON implements json.Marshaler.
NOTE: It adds an empty config object to the manifest, which is required by the registry, but not used by the client. In the future, the config object will not be required by the registry and this will should be removed.
type PushParams ¶
type PushParams struct {
// From is an optional destination name for the model. If empty, the
// destination name is the same as the source name.
From string
}
type Registry ¶
type Registry struct {
// Cache is the cache used to store models. If nil, [DefaultCache] is
// used.
Cache *blob.DiskCache
// UserAgent is the User-Agent header to send with requests to the
// registry. If empty, the User-Agent is determined by HTTPClient.
UserAgent string
// Key is the key used to authenticate with the registry.
//
// Currently, only Ed25519 keys are supported.
Key crypto.PrivateKey
// HTTPClient is the HTTP client used to make requests to the registry.
//
// If nil, [http.DefaultClient] is used.
//
// As a quick note: If a Registry function that makes a call to a URL
// with the "https+insecure" scheme, the client will be cloned and the
// transport will be set to skip TLS verification, unless the client's
// Transport done not have a Clone method with the same signature as
// [http.Transport.Clone], which case, the call will fail.
HTTPClient *http.Client
// MaxStreams is the maximum number of concurrent streams to use when
// pushing or pulling models. If zero, the number of streams is
// determined by [runtime.GOMAXPROCS].
//
// A negative value means no limit.
MaxStreams int
// ChunkingThreshold is the maximum size of a layer to download in a single
// request. If zero, [DefaultChunkingThreshold] is used.
ChunkingThreshold int64
// Mask, if set, is the name used to convert non-fully qualified names
// to fully qualified names.
// If empty, [DefaultMask] is used.
Mask string
// ReadTimeout is the maximum duration for reading the entire request,
// including the body.
// A zero or negative value means there will be no timeout.
ReadTimeout time.Duration
}
Registry is a client for performing push and pull operations against an Ollama registry.
Example (CancelOnFirstError) ¶
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
ctx = WithTrace(ctx, &Trace{
Update: func(l *Layer, n int64, err error) {
if err != nil {
// Discontinue pulling layers if there is an
// error instead of continuing to pull more
// data.
cancel()
}
},
})
var r Registry
if err := r.Pull(ctx, "model"); err != nil {
// panic for demo purposes
panic(err)
}
func DefaultRegistry ¶
DefaultRegistry returns a new Registry configured from the environment. The key is read from $HOME/.ollama/id_ed25519, MaxStreams is set to the value of OLLAMA_REGISTRY_MAXSTREAMS, and ReadTimeout is set to 30 seconds.
It returns an error if any configuration in the environment is invalid.
func (*Registry) Pull ¶
Pull pulls the model with the given name from the remote registry into the cache.
For layers larger then [Registry.MaxChunkSize], the layer is downloaded in chunks of the specified size, and then reassembled and verified. This is typically slower than splitting the model up across layers, and is mostly utilized for layers of type equal to "application/vnd.ollama.image".
func (*Registry) ResolveLocal ¶
ResolveLocal resolves a name to a Manifest in the local cache.
type Trace ¶
type Trace struct {
// Update is called during [Registry.Push] and [Registry.Pull] to
// report the progress of blob uploads and downloads.
//
// The n argument is the number of bytes transferred so far, and err is
// any error that has occurred. If n == 0, and err is nil, the download
// or upload has just started. If err is [ErrCached], the download or
// upload has been skipped because the blob is already present in the
// local cache or remote registry, respectively. Otherwise, if err is
// non-nil, the download or upload has failed. When l.Size == n, and
// err is nil, the download or upload has completed.
//
// A function assigned must be safe for concurrent use. The function is
// called synchronously and so should not block or take long to run.
Update func(_ *Layer, n int64, _ error)
}
Trace is a set of functions that are called to report progress during blob downloads and uploads.
Use WithTrace to attach a Trace to a context for use with Registry.Push and Registry.Pull.