schwift

package module
v2.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 8, 2024 License: Apache-2.0 Imports: 26 Imported by: 3

README

Schwift

GoDoc

This is a Go client library for OpenStack Swift. I made this after growing frustrated with the inflexible API design of ncw/swift; see near the bottom for details.

Installation

You can get this with go get github.com/majewsky/schwift. When using this in an application, vendoring is recommended.

Usage

This library uses Gophercloud to handle authentication, so to use Schwift, you have to first build a gophercloud.ServiceClient and then pass that to gopherschwift.Wrap() to get a handle on the Swift account.

For example, to connect to Swift using OpenStack Keystone authentication:

import (
  "github.com/gophercloud/gophercloud"
  "github.com/gophercloud/gophercloud/openstack"
  "github.com/majewsky/schwift/gopherschwift"
)

authOptions, err := openstack.AuthOptionsFromEnv()
provider, err := openstack.AuthenticatedClient(authOptions)
client, err := openstack.NewObjectStorageV1(provider, gophercloud.EndpointOpts{})

account, err := gopherschwift.Wrap(client, nil)

To connect to Swift using Swift's built-in authentication:

import (
  "github.com/gophercloud/gophercloud/openstack"
  "github.com/gophercloud/gophercloud/openstack/objectstore/v1/swauth"
  "github.com/majewsky/schwift/gopherschwift"
)

provider, err := openstack.NewClient("http://swift.example.com:8080")
client, err := swauth.NewObjectStorageV1(provider, swauth.AuthOpts {
    User: "project:user",
    Key:  "password",
})

account, err := gopherschwift.Wrap(client, nil)

From this point, follow the API documentation for what you can do with the schwift.Account object. For example, to download an object's contents into a string:

text, err := account.Container("foo").Object("bar.txt").Download(nil).AsString()

Why another Swift client library?

The most popular Swift client library is ncw/swift. I have used it extensively and my main gripe with it is that its API is mostly based on single functions. When your API is a function, you cannot easily add further arguments to it without breaking backwards compatibility. Whenever someone wants to do something slightly different, an entirely new function needs to be added. To witness, ncw/swift has five functions for listing objects, four functions for downloading objects, and three functions for uploading objects. (And that's without considering the separate API for large objects.) And still, when you try to do something that's not one of the 10 most common things, you're going to run into dead ends where the API does not allow you do specify that one URL parameter that you need. Like that one day when I filed five issues in a row because every function in the API that I tried turned out to be missing something.

Schwift improves on ncw/swift by:

  • allowing the user to set arbitrary headers and URL parameters in every request method,
  • including a pointer to RequestOpts in every request method, which can later be extended with new members without breaking backwards compatibility, and
  • providing a generic Request.Do() method as a last resort for users who need to do a request that absolutely cannot be made with the existing request methods.
What about Gophercloud?

Schwift uses Gophercloud for authentication. That solves one problem that ncw/swift has, namely that you cannot use the Keystone token that ncw/swift fetches for talking to other OpenStack services.

But besides the auth code, Schwift avoids all other parts of Gophercloud. Gophercloud, like many other OpenStack client libraries, is modeled frankly around the "JSON-in, JSON-out" request-response-based design that all OpenStack APIs share. All of them, except for Swift. A lot of the infrastructure that Gophercloud provides is not suited for Swift, mostly on account of it not using JSON bodies anywhere.

Furthermore, the API of Gophercloud is modeled around individual requests and responses, which means that there will probably never be support for advanced features like large objects unless you're willing to do all the footwork yourself.

Schwift improves on Gophercloud by providing a object-oriented API that respects and embraces Swift's domain model and API design.

Documentation

Overview

Package schwift is a client library for OpenStack Swift (https://github.com/openstack/swift, https://openstack.org).

Authentication with Gophercloud

Schwift does not implement authentication (neither Keystone nor Swift v1), but can be plugged into any library that does. The most common choice is Gophercloud (https://github.com/gophercloud/gophercloud/v2).

When using Gophercloud, you usually start by obtaining a gophercloud.ServiceClient for Swift like so: 

import (
	"github.com/gophercloud/gophercloud/v2/openstack"
	"github.com/gophercloud/utils/v2/openstack/clientconfig"
)

//option 1: build a gophercloud.AuthOptions instance yourself
provider, err := openstack.AuthenticatedClient(authOptions)
client, err := openstack.NewObjectStorageV1(provider, gophercloud.EndpointOpts{})

//option 2: have Gophercloud read the standard OS_* environment variables
provider, err := clientConfig.AuthenticatedClient(nil)
client, err := openstack.NewObjectStorageV1(provider, gophercloud.EndpointOpts{})

//option 3: if you're using Swift's builtin authentication instead of Keystone
provider, err := openstack.NewClient("http://swift.example.com:8080")
client, err := swauth.NewObjectStorageV1(provider, swauth.AuthOpts {
	User: "project:user",
	Key:  "password",
})

Then, in all these cases, you use gopherschwift to convert the gophercloud.ServiceClient into a schwift.Account instance, from which point you have access to all of schwift's API: 

import "github.com/majewsky/schwift/v2/gopherschwift"

account, err := gopherschwift.Wrap(client)

For example, to download an object's contents into a string: 

text, err := account.Container("foo").Object("bar.txt").Download(nil).AsString()

Authentication with a different OpenStack library

If you use a different Go library to handle Keystone/Swift authentication, take the client object that it provides and wrap it into something that implements the schwift.Backend interface. Then use schwift.InitializeAccount() to obtain a schwift.Account.

Caching

When a GET or HEAD request is sent by an Account, Container or Object instance, the headers associated with that thing will be stored in that instance and not retrieved again. 

obj := account.Container("foo").Object("bar")

hdr, err := obj.Headers() //sends HTTP request "HEAD <storage-url>/foo/bar"
...
hdr, err = obj.Headers()  //returns cached values immediately

If this behavior is not desired, the Invalidate() method can be used to clear caches on any Account, Container or Object instance. Some methods that modify the instance on the server call Invalidate() automatically, e.g. Object.Upload(), Update() or Delete(). This will be indicated in the method's documentation.

Error handling

When a method on an Account, Container or Object instance makes a HTTP request to Swift and Swift returns an unexpected status code, a schwift.UnexpectedStatusCodeError will be returned. Schwift provides the convenience function Is() to check the status code of these errors to detect common failure situations: 

obj := account.Container("foo").Object("bar")
err := obj.Upload(bytes.NewReader(data), nil)

if schwift.Is(err, http.StatusRequestEntityTooLarge) {
	log.Print("quota exceeded for container foo!")
} else if err != nil {
	log.Fatal("unexpected error: " + err.Error())
}

The documentation for a method may indicate certain common error conditions that can be detected this way by stating that "This method fails with http.StatusXXX if ...". Because of the wide variety of failure modes in Swift, this information is not guaranteed to be exhaustive.

Index

Constants

View Source 
const DefaultUserAgent = "schwift/" + Version

DefaultUserAgent is the User-Agent string that Backend implementations should use if the user does not provide their own User-Agent string.

View Source 
const Version = "1.0.0"

Version contains the version number of Schwift.

Variables

View Source 
var (
	// ErrChecksumMismatch is returned by Object.Upload() when the Etag in the
	// server response does not match the uploaded data.
	ErrChecksumMismatch = errors.New("Etag on uploaded object does not match MD5 checksum of uploaded data")
	// ErrNoContainerName is returned by Request.Do() if ObjectName is given, but
	// ContainerName is empty.
	ErrNoContainerName = errors.New("missing container name")
	// ErrMalformedContainerName is returned by Request.Do() if ContainerName
	// contains slashes.
	ErrMalformedContainerName = errors.New("container name may not contain slashes")
	// ErrNotSupported is returned by bulk operations, large object operations,
	// etc. if the server does not support the requested operation.
	ErrNotSupported = errors.New("operation not supported by this Swift server")
	// ErrAccountMismatch is returned by operations on an account that accept
	// containers/objects as arguments, if some or all of the provided
	// containers/objects are located in a different account.
	ErrAccountMismatch = errors.New("some of the given objects are not in this account")
	// ErrContainerMismatch is returned by operations on a container that accept
	// objects as arguments, if some or all of the provided objects are located in
	// a different container.
	ErrContainerMismatch = errors.New("some of the given objects are not in this container")
	// ErrNotLarge is returned by Object.AsLargeObject() if the object does not
	// exist, or if it is not a large object composed out of segments.
	ErrNotLarge = errors.New("not a large object")
	// ErrSegmentInvalid is returned by LargeObject.AddSegment() if the segment
	// provided is malformed or uses features not supported by the LargeObject's
	// strategy. See documentation for LargeObject.AddSegment() for details.
	ErrSegmentInvalid = errors.New("segment invalid or incompatible with large object strategy")
)

Functions

func Is 

func Is(err error, code int) bool

Is checks if the given error is an UnexpectedStatusCodeError for that status code. For example: 

err := container.Delete(nil)
if err != nil {
    if schwift.Is(err, http.StatusNotFound) {
        //container does not exist -> just what we wanted
        return nil
    } else {
        //report unexpected error
        return err
    }
}

It is safe to pass a nil error, in which case Is() always returns false.

Types

type Account 

type Account struct {
	// contains filtered or unexported fields
}

Account represents a Swift account. Instances are usually obtained by connecting to a backend (see package-level documentation), or by traversing upwards from a container with Container.Account().

func InitializeAccount 

func InitializeAccount(backend Backend) (*Account, error)

InitializeAccount takes something that implements the Backend interface, and returns the Account instance corresponding to the account/project that this backend is connected to.

func (*Account) Backend 

func (a *Account) Backend() Backend

Backend returns the backend which is used to make requests against this account.

func (*Account) BulkDelete 

func (a *Account) BulkDelete(ctx context.Context, objects []*Object, containers []*Container, opts *RequestOptions) (numDeleted, numNotFound int, deleteError error)

BulkDelete deletes a large number of objects (and containers) at once. Containers are queued at the end of the deletion, so a container can be deleted in the same call in which all objects in it are deleted.

For example, to delete all objects in a container: 

var container *schwift.Container

objects, err := container.Objects().Collect()
numDeleted, numNotFound, err := container.Account().BulkDelete(objects, nil, nil)

To also delete the container: 

var container *schwift.Container

objects, err := container.Objects().Collect()
numDeleted, numNotFound, err := container.Account().BulkDelete(
    objects, []*schwift.Container{container}, nil)

If the server does not support bulk-deletion, this function falls back to deleting each object and container individually, and aggregates the result.

If not nil, the error return value is *usually* an instance of BulkError.

The objects may be located in multiple containers, but they and the containers must all be located in the given account. (Otherwise, ErrAccountMismatch is returned.)

func (*Account) BulkUpload 

func (a *Account) BulkUpload(ctx context.Context, uploadPath string, format BulkUploadFormat, contents io.Reader, opts *RequestOptions) (int, error)

BulkUpload extracts an archive (which may contain multiple files) into a Swift account. The path of each file in the archive is appended to the uploadPath to form the FullName() of the resulting Object.

For example, when uploading an archive that contains the file "a/b/c": 

//This uploads the file into the container "a" as object "b/c".
account.BulkUpload("", format, contents, nil)
//This uploads the file into the container "foo" as object "a/b/c".
account.BulkUpload("foo", format, contents, nil)
//This uploads the file into the container "foo" as object "bar/baz/a/b/c".
account.BulkUpload("foo/bar/baz", format, contents, nil)

The first return value indicates the number of files that have been created on the server side. This may be lower than the number of files in the archive if some files could not be saved individually (e.g. because a quota was exceeded in the middle of the archive extraction).

If not nil, the error return value is *usually* an instance of BulkError.

This operation returns (0, ErrNotSupported) if the server does not support bulk-uploading.

func (*Account) Capabilities 

func (a *Account) Capabilities(ctx context.Context) (Capabilities, error)

Capabilities queries the GET /info endpoint of the Swift server providing this account. Capabilities are cached, so the GET request will only be sent once during the first call to this method.

func (*Account) Container 

func (a *Account) Container(name string) *Container

Container returns a handle to the container with the given name within this account. This function does not issue any HTTP requests, and therefore cannot ensure that the container exists. Use the Exists() function to check for the container's existence, or chain this function with the EnsureExists() function like so: 

container, err := account.Container("documents").EnsureExists()

func (*Account) Containers 

func (a *Account) Containers() *ContainerIterator

Containers returns a ContainerIterator that lists the containers in this account. The most common use case is: 

containers, err := account.Containers().Collect()

You can extend this by configuring the iterator before collecting the results: 

iter := account.Containers()
iter.Prefix = "test-"
containers, err := iter.Collect()

Or you can use a different iteration method: 

err := account.Containers().ForeachDetailed(func (ci ContainerInfo) error {
	log.Printf("container %s contains %d objects!\n",
		ci.Container.Name(), ci.ObjectCount)
})

func (*Account) Create 

func (a *Account) Create(ctx context.Context, opts *RequestOptions) error

Create creates the account using a PUT request. This operation is only available to reseller admins, not to regular users.

A successful PUT request implies Invalidate() since it may change metadata.

func (*Account) Headers 

func (a *Account) Headers(ctx context.Context) (AccountHeaders, error)

Headers returns the AccountHeaders for this account. If the AccountHeaders has not been cached yet, a HEAD request is issued on the account.

This operation fails with http.StatusNotFound if the account does not exist.

WARNING: This method is not thread-safe. Calling it concurrently on the same object results in undefined behavior.

func (*Account) Invalidate 

func (a *Account) Invalidate()

Invalidate clears the internal cache of this Account instance. The next call to Headers() on this instance will issue a HEAD request on the account.

WARNING: This method is not thread-safe. Calling it concurrently on the same object results in undefined behavior.

func (*Account) IsEqualTo 

func (a *Account) IsEqualTo(other *Account) bool

IsEqualTo returns true if both Account instances refer to the same account.

func (*Account) Name 

func (a *Account) Name() string

Name returns the name of the account (usually the prefix "AUTH_" followed by the Keystone project ID).

func (*Account) RawCapabilities 

func (a *Account) RawCapabilities(ctx context.Context) ([]byte, error)

RawCapabilities queries the GET /info endpoint of the Swift server providing this account, and returns the response body. Unlike Account.Capabilities, this method does not employ any caching.

func (*Account) SwitchAccount 

func (a *Account) SwitchAccount(accountName string) *Account

SwitchAccount returns a handle to a different account on the same server. Note that you need reseller permissions to access accounts other than that where you originally authenticated. This method does not check whether the account actually exists.

The account name is usually the Keystone project ID with an additional "AUTH_" prefix.

func (*Account) Update 

func (a *Account) Update(ctx context.Context, headers AccountHeaders, opts *RequestOptions) error

Update updates the account using a POST request. The headers in the headers attribute take precedence over those in opts.Headers.

A successful POST request implies Invalidate() since it may change metadata.

type AccountHeaders 

type AccountHeaders struct {
	Headers
}

AccountHeaders contains the headers for a schwift.Account instance.

To read and write well-known headers, use the methods on this type. To read and write arbitrary headers, use the methods on the Headers supertype.

func NewAccountHeaders 

func NewAccountHeaders() AccountHeaders

NewAccountHeaders creates a new AccountHeaders instance. The return value will have the Headers attribute initialized to a non-nil map.

func (AccountHeaders) BytesUsed 

func (h AccountHeaders) BytesUsed() FieldUint64Readonly

BytesUsed provides type-safe access to X-Account-Bytes-Used headers.

func (AccountHeaders) BytesUsedQuota 

func (h AccountHeaders) BytesUsedQuota() FieldUint64

BytesUsedQuota provides type-safe access to X-Account-Meta-Quota-Bytes headers.

func (AccountHeaders) ContainerCount 

func (h AccountHeaders) ContainerCount() FieldUint64Readonly

ContainerCount provides type-safe access to X-Account-Container-Count headers.

func (AccountHeaders) CreatedAt 

func (h AccountHeaders) CreatedAt() FieldUnixTimeReadonly

CreatedAt provides type-safe access to X-Timestamp headers.

func (AccountHeaders) Metadata 

func (h AccountHeaders) Metadata() FieldMetadata

Metadata provides type-safe access to X-Account-Meta- headers.

func (AccountHeaders) ObjectCount 

func (h AccountHeaders) ObjectCount() FieldUint64Readonly

ObjectCount provides type-safe access to X-Account-Object-Count headers.

func (AccountHeaders) TempURLKey 

func (h AccountHeaders) TempURLKey() FieldString

TempURLKey provides type-safe access to X-Account-Meta-Temp-URL-Key headers.

func (AccountHeaders) TempURLKey2 

func (h AccountHeaders) TempURLKey2() FieldString

TempURLKey2 provides type-safe access to X-Account-Meta-Temp-URL-Key-2 headers.

func (AccountHeaders) Validate 

func (h AccountHeaders) Validate() error

Validate returns MalformedHeaderError if the value of any well-known header does not conform to its data type. This is called automatically by Schwift when preparing an AccountHeaders instance from a GET/HEAD response, so you usually do not need to do it yourself. You will get the validation error from the Account method doing the request, e.g. Headers().

type Backend 

type Backend interface {
	// EndpointURL returns the endpoint URL from the Keystone catalog for the
	// Swift account that this backend operates on. It should look like
	// `http://domain.tld/v1/AUTH_projectid/`. The trailing slash is required.
	EndpointURL() string
	// Clone returns a deep clone of this backend with the endpoint URL changed to
	// the given URL. This is used by Account.SwitchAccount().
	Clone(newEndpointURL string) Backend
	// Do executes the given HTTP request after adding to it the X-Auth-Token
	// header containing the backend's current Keystone (or Swift auth) token. If
	// the status code returned is 401, it shall attempt to acquire a new auth
	// token and restart the request with the new token.
	//
	// If the user has not supplied their own User-Agent string to the backend,
	// the backend should use the schwift.DefaultUserAgent constant instead.
	Do(req *http.Request) (*http.Response, error)
}

Backend is the interface between Schwift and the libraries providing authentication for it. Each instance of Backend represents a particular Swift account.

type BulkError 

type BulkError struct {
	// StatusCode contains the overall HTTP status code of the operation.
	StatusCode int
	// OverallError contains the fatal error that aborted the bulk operation, or a
	// summary of which recoverable errors were encountered. It may be empty.
	OverallError string
	// ObjectErrors contains errors that occurred while working on individual
	// objects or containers. It may be empty if no such errors occurred.
	ObjectErrors []BulkObjectError
}

BulkError is returned by Account.BulkUpload() when the archive was uploaded and unpacked successfully, but some (or all) objects could not be saved in Swift; and by Account.BulkDelete() when not all requested objects could be deleted.

func (BulkError) Error 

func (e BulkError) Error() string

Error implements the builtin/error interface. To fit into one line, it condenses the ObjectErrors into a count.

type BulkObjectError 

type BulkObjectError struct {
	ContainerName string
	ObjectName    string
	StatusCode    int
}

BulkObjectError is the error message for a single object in a bulk operation. It is not generated individually, only as part of BulkError.

func (BulkObjectError) Error 

func (e BulkObjectError) Error() string

Error implements the builtin/error interface.

type BulkUploadFormat 

type BulkUploadFormat string

BulkUploadFormat enumerates possible archive formats for Container.BulkUpload(). 

const (
	// BulkUploadTar is a plain tar archive.
	BulkUploadTar BulkUploadFormat = "tar"
	// BulkUploadTarGzip is a GZip-compressed tar archive.
	BulkUploadTarGzip BulkUploadFormat = "tar.gz"
	// BulkUploadTarBzip2 is a BZip2-compressed tar archive.
	BulkUploadTarBzip2 BulkUploadFormat = "tar.bz2"
)

type Capabilities 

type Capabilities struct {
	BulkDelete *struct {
		MaximumDeletesPerRequest uint `json:"max_deletes_per_request"`
		MaximumFailedDeletes     uint `json:"max_failed_deletes"`
	} `json:"bulk_delete"`
	BulkUpload *struct {
		MaximumContainersPerExtraction uint `json:"max_containers_per_extraction"`
		MaximumFailedExtractions       uint `json:"max_failed_extractions"`
	} `json:"bulk_upload"`
	StaticLargeObject *struct {
		MaximumManifestSegments uint `json:"max_manifest_segments"`
		MaximumManifestSize     uint `json:"max_manifest_size"`
		MinimumSegmentSize      uint `json:"min_segment_size"`
	} `json:"slo"`
	Swift struct {
		AccountAutocreate          bool                `json:"account_autocreate"`
		AccountListingLimit        uint                `json:"account_listing_limit"`
		AllowAccountManagement     bool                `json:"allow_account_management"`
		ContainerListingLimit      uint                `json:"container_listing_limit"`
		ExtraHeaderCount           uint                `json:"extra_header_count"`
		MaximumAccountNameLength   uint                `json:"max_account_name_length"`
		MaximumContainerNameLength uint                `json:"max_container_name_length"`
		MaximumFileSize            uint                `json:"max_file_size"`
		MaximumHeaderSize          uint                `json:"max_header_size"`
		MaximumMetaCount           uint                `json:"max_meta_count"`
		MaximumMetaNameLength      uint                `json:"max_meta_name_length"`
		MaximumMetaOverallSize     uint                `json:"max_meta_overall_size"`
		MaximumMetaValueLength     uint                `json:"max_meta_value_length"`
		MaximumObjectNameLength    uint                `json:"max_object_name_length"`
		Policies                   []StoragePolicySpec `json:"policies"`
		StrictCORSMode             bool                `json:"strict_cors_mode"`
		Version                    string              `json:"version"`
	} `json:"swift"`
	Swift3 *struct {
		AllowMultipartUploads     bool   `json:"allow_multipart_uploads"`
		MaximumBucketListing      uint   `json:"max_bucket_listing"`
		MaximumMultiDeleteObjects uint   `json:"max_multi_delete_objects"`
		MaximumPartsListing       uint   `json:"max_parts_listing"`
		MaximumUploadPartNumber   uint   `json:"max_upload_part_num"`
		Version                   string `json:"version"`
	} `json:"swift3"`
	Symlink *struct {
		MaximumLoopCount uint `json:"symloop_max"`
	} `json:"symlink"`
	TempAuth *struct {
		AccountACLs bool `json:"account_acls"`
	} `json:"tempauth"`
	TempURL *struct {
		AllowedDigests        []string `json:"allowed_digests"`
		IncomingAllowHeaders  []string `json:"incoming_allow_headers"`
		IncomingRemoveHeaders []string `json:"incoming_remove_headers"`
		Methods               []string `json:"methods"`
		OutgoingAllowHeaders  []string `json:"outgoing_allow_headers"`
		OutgoingRemoveHeaders []string `json:"outgoing_remove_headers"`
	} `json:"tempurl"`
}

Capabilities describes a subset of the capabilities that Swift can report under its /info endpoint. This struct is obtained through the Account.Capabilities() method. To query capabilities not represented in this struct, see Account.QueryCapabilities().

All direct members of struct Capabilities, except for "Swift", are pointers. If any of these is nil, it indicates that the middleware corresponding to that field is not available on this server.

type Container 

type Container struct {
	// contains filtered or unexported fields
}

Container represents a Swift container. Instances are usually obtained by traversing downwards from an account with Account.Container() or Account.Containers(), or upwards from an object with Object.Container().

func (*Container) Account 

func (c *Container) Account() *Account

Account returns a handle to the account this container is stored in.

func (*Container) Create 

func (c *Container) Create(ctx context.Context, opts *RequestOptions) error

Create creates the container using a PUT request. To add URL parameters, pass a non-nil *RequestOptions.

This function can be used regardless of whether the container exists or not.

A successful PUT request implies Invalidate() since it may change metadata.

func (*Container) Delete 

func (c *Container) Delete(ctx context.Context, opts *RequestOptions) error

Delete deletes the container using a DELETE request. To add URL parameters, pass a non-nil *RequestOptions.

This operation fails with http.StatusConflict if the container is not empty.

This operation fails with http.StatusNotFound if the container does not exist.

A successful DELETE request implies Invalidate().

func (*Container) EnsureExists 

func (c *Container) EnsureExists(ctx context.Context) (*Container, error)

EnsureExists issues a PUT request on this container. If the container does not exist yet, it will be created by this call. If the container exists already, this call does not change it. This function returns the same container again, because its intended use is with freshly constructed Container instances like so: 

container, err := account.Container("documents").EnsureExists()

func (*Container) Exists 

func (c *Container) Exists(ctx context.Context) (bool, error)

Exists checks if this container exists, potentially by issuing a HEAD request if no Headers() have been cached yet.

func (*Container) Headers 

func (c *Container) Headers(ctx context.Context) (ContainerHeaders, error)

Headers returns the ContainerHeaders for this container. If the ContainerHeaders has not been cached yet, a HEAD request is issued on the container.

This operation fails with http.StatusNotFound if the container does not exist.

WARNING: This method is not thread-safe. Calling it concurrently on the same object results in undefined behavior.

func (*Container) Invalidate 

func (c *Container) Invalidate()

Invalidate clears the internal cache of this Container instance. The next call to Headers() on this instance will issue a HEAD request on the container.

WARNING: This method is not thread-safe. Calling it concurrently on the same object results in undefined behavior.

func (*Container) IsEqualTo 

func (c *Container) IsEqualTo(other *Container) bool

IsEqualTo returns true if both Container instances refer to the same container.

func (*Container) Name 

func (c *Container) Name() string

Name returns the container name.

func (*Container) Object 

func (c *Container) Object(name string) *Object

Object returns a handle to the object with the given name within this container. This function does not issue any HTTP requests, and therefore cannot ensure that the object exists. Use the Exists() function to check for the object's existence.

func (*Container) Objects 

func (c *Container) Objects() *ObjectIterator

Objects returns an ObjectIterator that lists the objects in this container. The most common use case is: 

objects, err := container.Objects().Collect()

You can extend this by configuring the iterator before collecting the results: 

iter := container.Objects()
iter.Prefix = "test-"
objects, err := iter.Collect()

Or you can use a different iteration method: 

err := container.Objects().ForeachDetailed(func (info ObjectInfo) error {
    log.Printf("object %s is %d bytes large!\n",
        info.Object.Name(), info.SizeBytes)
})

func (*Container) URL 

func (c *Container) URL() (string, error)

URL returns the canonical URL for this container on the server. This is particularly useful when the ReadACL on the account or container is set to allow anonymous read access.

func (*Container) Update 

func (c *Container) Update(ctx context.Context, headers ContainerHeaders, opts *RequestOptions) error

Update updates the container using a POST request. To add URL parameters, pass a non-nil *RequestOptions.

If you are not sure whether the container exists, use Create() instead.

A successful POST request implies Invalidate() since it may change metadata.

type ContainerHeaders 

type ContainerHeaders struct {
	Headers
}

ContainerHeaders contains the headers for a schwift.Container instance.

To read and write well-known headers, use the methods on this type. To read and write arbitrary headers, use the methods on the Headers supertype.

func NewContainerHeaders 

func NewContainerHeaders() ContainerHeaders

NewContainerHeaders creates a new ContainerHeaders instance. The return value will have the Headers attribute initialized to a non-nil map.

func (ContainerHeaders) BytesUsed 

func (h ContainerHeaders) BytesUsed() FieldUint64Readonly

BytesUsed provides type-safe access to X-Container-Bytes-Used headers.

func (ContainerHeaders) BytesUsedQuota 

func (h ContainerHeaders) BytesUsedQuota() FieldUint64

BytesUsedQuota provides type-safe access to X-Container-Meta-Quota-Bytes headers.

func (ContainerHeaders) CreatedAt 

func (h ContainerHeaders) CreatedAt() FieldUnixTimeReadonly

CreatedAt provides type-safe access to X-Timestamp headers.

func (ContainerHeaders) HistoryLocation 

func (h ContainerHeaders) HistoryLocation() FieldString

HistoryLocation provides type-safe access to X-History-Location headers.

func (ContainerHeaders) Metadata 

func (h ContainerHeaders) Metadata() FieldMetadata

Metadata provides type-safe access to X-Container-Meta- headers.

func (ContainerHeaders) ObjectCount 

func (h ContainerHeaders) ObjectCount() FieldUint64Readonly

ObjectCount provides type-safe access to X-Container-Object-Count headers.

func (ContainerHeaders) ObjectCountQuota 

func (h ContainerHeaders) ObjectCountQuota() FieldUint64

ObjectCountQuota provides type-safe access to X-Container-Meta-Quota-Count headers.

func (ContainerHeaders) ReadACL 

func (h ContainerHeaders) ReadACL() FieldString

ReadACL provides type-safe access to X-Container-Read headers.

func (ContainerHeaders) StoragePolicy 

func (h ContainerHeaders) StoragePolicy() FieldString

StoragePolicy provides type-safe access to X-Storage-Policy headers.

func (ContainerHeaders) SyncKey 

func (h ContainerHeaders) SyncKey() FieldString

SyncKey provides type-safe access to X-Container-Sync-Key headers.

func (ContainerHeaders) SyncTo 

func (h ContainerHeaders) SyncTo() FieldString

SyncTo provides type-safe access to X-Container-Sync-To headers.

func (ContainerHeaders) TempURLKey 

func (h ContainerHeaders) TempURLKey() FieldString

TempURLKey provides type-safe access to X-Container-Meta-Temp-URL-Key headers.

func (ContainerHeaders) TempURLKey2 

func (h ContainerHeaders) TempURLKey2() FieldString

TempURLKey2 provides type-safe access to X-Container-Meta-Temp-URL-Key-2 headers.

func (ContainerHeaders) Validate 

func (h ContainerHeaders) Validate() error

Validate returns MalformedHeaderError if the value of any well-known header does not conform to its data type. This is called automatically by Schwift when preparing an ContainerHeaders instance from a GET/HEAD response, so you usually do not need to do it yourself. You will get the validation error from the Container method doing the request, e.g. Headers().

func (ContainerHeaders) VersionsLocation 

func (h ContainerHeaders) VersionsLocation() FieldString

VersionsLocation provides type-safe access to X-Versions-Location headers.

func (ContainerHeaders) WriteACL 

func (h ContainerHeaders) WriteACL() FieldString

WriteACL provides type-safe access to X-Container-Write headers.

type ContainerInfo 

type ContainerInfo struct {
	Container    *Container
	ObjectCount  uint64
	BytesUsed    uint64
	LastModified time.Time
}

ContainerInfo is a result type returned by ContainerIterator for detailed container listings. The metadata in this type is a subset of Container.Headers(), but since it is returned as part of the detailed container listing, it can be obtained without making additional HEAD requests on the container(s).

type ContainerIterator 

type ContainerIterator struct {
	Account *Account
	// When Prefix is set, only containers whose name starts with this string are
	// returned.
	Prefix string
	// Options may contain additional headers and query parameters for the GET request.
	Options *RequestOptions
	// contains filtered or unexported fields
}

ContainerIterator iterates over the accounts in a container. It is typically constructed with the Account.Containers() method. For example: 

//either this...
iter := account.Containers()
iter.Prefix = "test-"
containers, err := iter.Collect()

//...or this
containers, err := schwift.ContainerIterator{
	Account: account,
	Prefix: "test-",
}.Collect()

When listing containers via a GET request on the account, you can choose to receive container names only (via the methods without the "Detailed" suffix), or container names plus some basic metadata fields (via the methods with the "Detailed" suffix). See struct ContainerInfo for which metadata is returned.

To obtain any other metadata, you can call Container.Headers() on the result container, but this will issue a separate HEAD request for each container.

Use the "Detailed" methods only when you use the extra metadata in struct ContainerInfo; detailed GET requests are more expensive than simple ones that return only container names.

func (*ContainerIterator) Collect 

func (i *ContainerIterator) Collect(ctx context.Context) ([]*Container, error)

Collect lists all container names matching this iterator. For large sets of containers that cannot be retrieved at once, Collect handles paging behind the scenes. The return value is always the complete set of containers.

func (*ContainerIterator) CollectDetailed 

func (i *ContainerIterator) CollectDetailed(ctx context.Context) ([]ContainerInfo, error)

CollectDetailed is like Collect, but includes basic metadata.

func (*ContainerIterator) Foreach 

func (i *ContainerIterator) Foreach(ctx context.Context, callback func(*Container) error) error

Foreach lists the container names matching this iterator and calls the callback once for every container. Iteration is aborted when a GET request fails, or when the callback returns a non-nil error.

func (*ContainerIterator) ForeachDetailed 

func (i *ContainerIterator) ForeachDetailed(ctx context.Context, callback func(ContainerInfo) error) error

ForeachDetailed is like Foreach, but includes basic metadata.

func (*ContainerIterator) NextPage 

func (i *ContainerIterator) NextPage(ctx context.Context, limit int) ([]*Container, error)

NextPage queries Swift for the next page of container names. If limit is >= 0, not more than that many container names will be returned at once. Note that the server also has a limit for how many containers to list in one request; the lower limit wins.

The end of the container listing is reached when an empty list is returned.

This method offers maximal flexibility, but most users will prefer the simpler interfaces offered by Collect() and Foreach().

func (*ContainerIterator) NextPageDetailed 

func (i *ContainerIterator) NextPageDetailed(ctx context.Context, limit int) ([]ContainerInfo, error)

NextPageDetailed is like NextPage, but includes basic metadata.

type CopyOptions 

type CopyOptions struct {
	// Copy only the object's content, not its metadata. New metadata can always
	// be supplied in the RequestOptions argument of Object.CopyTo().
	FreshMetadata bool
	// When the source is a symlink, copy the symlink instead of the target object.
	ShallowCopySymlinks bool
}

CopyOptions invokes advanced behavior in the Object.Copy() method.

type DeleteOptions 

type DeleteOptions struct {
	// When deleting a large object, also delete its segments. This will cause
	// Delete() to call into BulkDelete(), so a BulkError may be returned.
	DeleteSegments bool
}

DeleteOptions invokes advanced behavior in the Object.Delete() method.

type DownloadedObject 

type DownloadedObject struct {
	// contains filtered or unexported fields
}

DownloadedObject is returned by Object.Download(). It wraps the io.ReadCloser from http.Response.Body with convenience methods for collecting the contents into a byte slice or string. 

var obj *swift.Object

//Do NOT do this!
reader, err := obj.Download(nil).AsReadCloser()
bytes, err := io.ReadAll(reader)
err := reader.Close()
str := string(bytes)

//Do this instead:
str, err := obj.Download(nil).AsString()

Since all methods on DownloadedObject are irreversible, the idiomatic way of using DownloadedObject is to call one of its members immediately, without storing the DownloadedObject instance in a variable first. 

var obj *swift.Object

//Do NOT do this!
downloaded := obj.Download(nil)
reader, err := downloaded.AsReadCloser()

//Do this instead:
reader, err := obj.Download(nil).AsReadCloser()

func (DownloadedObject) AsByteSlice 

func (o DownloadedObject) AsByteSlice() ([]byte, error)

AsByteSlice collects the contents of this downloaded object into a byte slice.

func (DownloadedObject) AsReadCloser 

func (o DownloadedObject) AsReadCloser() (io.ReadCloser, error)

AsReadCloser returns an io.ReadCloser containing the contents of the downloaded object.

func (DownloadedObject) AsString 

func (o DownloadedObject) AsString() (string, error)

AsString collects the contents of this downloaded object into a string.

type FieldHTTPTimeReadonly 

type FieldHTTPTimeReadonly struct {
	// contains filtered or unexported fields
}

FieldHTTPTimeReadonly is a helper type that provides type-safe access to a readonly Swift header whose value is a HTTP timestamp like this: 

Mon, 02 Jan 2006 15:04:05 GMT

It cannot be directly constructed, but methods on the Headers types return this type. For example: 

//suppose you have:
hdr, err := obj.Headers()

//you could do this:
time, err := time.Parse(time.RFC1123, hdr.Get("Last-Modified"))

//or you can just:
time := hdr.UpdatedAt().Get()

Don't worry about the missing `err` in the last line. When the header fails to parse, Object.Headers() already returns the corresponding MalformedHeaderError.

func (FieldHTTPTimeReadonly) Exists 

func (f FieldHTTPTimeReadonly) Exists() bool

Exists checks whether there is a value for this header.

func (FieldHTTPTimeReadonly) Get 

func (f FieldHTTPTimeReadonly) Get() time.Time

Get returns the value for this header, or the zero value if there is no value (or if it is not a valid timestamp).

type FieldMetadata 

type FieldMetadata struct {
	// contains filtered or unexported fields
}

FieldMetadata is a helper type that provides safe access to the metadata headers in a headers instance. It cannot be directly constructed, but each headers type has a method "Metadata" returning this type. For example: 

hdr := NewObjectHeaders()
//the following two statements are equivalent
hdr["X-Object-Meta-Access"] = "strictly confidential"
hdr.Metadata().Set("Access", "strictly confidential")

func (FieldMetadata) Clear 

func (m FieldMetadata) Clear(key string)

Clear works like Headers.Clear(), but prepends the metadata prefix to the key.

func (FieldMetadata) Del 

func (m FieldMetadata) Del(key string)

Del works like Headers.Del(), but prepends the metadata prefix to the key.

func (FieldMetadata) Get 

func (m FieldMetadata) Get(key string) string

Get works like Headers.Get(), but prepends the metadata prefix to the key.

func (FieldMetadata) Set 

func (m FieldMetadata) Set(key, value string)

Set works like Headers.Set(), but prepends the metadata prefix to the key.

type FieldString 

type FieldString struct {
	// contains filtered or unexported fields
}

FieldString is a helper type that provides type-safe access to a Swift header key whose value is a string. It cannot be directly constructed, but methods on the Headers types return this type. For example: 

hdr := NewAccountHeaders()
//the following two statements are equivalent:
hdr["X-Container-Read"] = ".r:*,.rlistings"
hdr.ReadACL().Set(".r:*,.rlistings")

func (FieldString) Clear 

func (f FieldString) Clear()

Clear sets this key to an empty string in the original headers instance, so that the key will be removed on the server during Update().

func (FieldString) Del 

func (f FieldString) Del()

Del removes this key from the original headers instance, so that the key will remain unchanged on the server during Update().

func (FieldString) Exists 

func (f FieldString) Exists() bool

Exists checks whether there is a value for this header.

func (FieldString) Get 

func (f FieldString) Get() string

Get returns the value for this header, or the empty string if there is no value.

func (FieldString) Set 

func (f FieldString) Set(value string)

Set writes a new value for this header into the corresponding headers instance.

type FieldUint64 

type FieldUint64 struct {
	// contains filtered or unexported fields
}

FieldUint64 is a helper type that provides type-safe access to a Swift header whose value is an unsigned integer. It cannot be directly constructed, but methods on the Headers types return this type. For example: 

hdr := NewAccountHeaders()
//the following two statements are equivalent:
hdr["X-Account-Meta-Quota-Bytes"] = "1048576"
hdr.BytesUsedQuota().Set(1 << 20)

func (FieldUint64) Clear 

func (f FieldUint64) Clear()

Clear sets this key to an empty string in the original headers instance, so that the key will be removed on the server during Update().

func (FieldUint64) Del 

func (f FieldUint64) Del()

Del removes this key from the original headers instance, so that the key will remain unchanged on the server during Update().

func (FieldUint64) Exists 

func (f FieldUint64) Exists() bool

Exists checks whether there is a value for this header.

func (FieldUint64) Get 

func (f FieldUint64) Get() uint64

Get returns the value for this header, or 0 if there is no value (or if it is not a valid uint64).

func (FieldUint64) Set 

func (f FieldUint64) Set(value uint64)

Set writes a new value for this header into the corresponding headers instance.

type FieldUint64Readonly 

type FieldUint64Readonly struct {
	// contains filtered or unexported fields
}

FieldUint64Readonly is a readonly variant of FieldUint64. It is used for fields that cannot be set by the client.

func (FieldUint64Readonly) Exists 

func (f FieldUint64Readonly) Exists() bool

Exists checks whether there is a value for this header.

func (FieldUint64Readonly) Get 

func (f FieldUint64Readonly) Get() uint64

Get returns the value for this header, or 0 if there is no value (or if it is not a valid uint64).

type FieldUnixTime 

type FieldUnixTime struct {
	// contains filtered or unexported fields
}

FieldUnixTime is a helper type that provides type-safe access to a Swift header whose value is a UNIX timestamp. It cannot be directly constructed, but methods on the Headers types return this type. For example: 

//suppose you have:
hdr, err := obj.Headers()

//you could do all this:
sec, err := strconv.ParseFloat(hdr.Get("X-Delete-At"), 64)
time := time.Unix(0, int64(1e9 * sec))

//or you can just:
time := hdr.ExpiresAt().Get()

Don't worry about the missing `err` in the last line. When the header fails to parse, Object.Headers() already returns the corresponding MalformedHeaderError.

func (FieldUnixTime) Clear 

func (f FieldUnixTime) Clear()

Clear sets this key to an empty string in the original headers instance, so that the key will be removed on the server during Update().

func (FieldUnixTime) Del 

func (f FieldUnixTime) Del()

Del removes this key from the original headers instance, so that the key will remain unchanged on the server during Update().

func (FieldUnixTime) Exists 

func (f FieldUnixTime) Exists() bool

Exists checks whether there is a value for this header.

func (FieldUnixTime) Get 

func (f FieldUnixTime) Get() time.Time

Get returns the value for this header, or the zero value if there is no value (or if it is not a valid timestamp).

func (FieldUnixTime) Set 

func (f FieldUnixTime) Set(value time.Time)

Set writes a new value for this header into the corresponding headers instance.

type FieldUnixTimeReadonly 

type FieldUnixTimeReadonly struct {
	// contains filtered or unexported fields
}

FieldUnixTimeReadonly is a readonly variant of FieldUnixTime. It is used for fields that cannot be set by the client.

func (FieldUnixTimeReadonly) Exists 

func (f FieldUnixTimeReadonly) Exists() bool

Exists checks whether there is a value for this header.

func (FieldUnixTimeReadonly) Get 

func (f FieldUnixTimeReadonly) Get() time.Time

Get returns the value for this header, or the zero value if there is no value (or if it is not a valid timestamp).

type Headers 

type Headers map[string]string

Headers represents a set of request headers or response headers.

Users will typically use one of the subtypes (AccountHeaders, ContainerHeaders, ObjectHeaders) instead, which provide type-safe access to well-known headers. The http.Header-like interface on this type can be used read and write arbitrary headers. For example, the following calls are equivalent: 

h := make(AccountHeaders)
h.Headers.Set("X-Account-Meta-Quota-Bytes", "1048576")
h.BytesUsedQuota().Set(1048576)

func (Headers) Clear 

func (h Headers) Clear(key string)

Clear sets the value for the specified header to the empty string. When the Headers instance is then sent to the server with Update(), the server will delete the value for that header; cf. Del().

func (Headers) Del 

func (h Headers) Del(key string)

Del deletes a key from the Headers instance. When the Headers instance is then sent to the server with Update(), a key which has been deleted with Del() will remain unchanged on the server.

For most writable attributes, a key which has been deleted with Del() will remain unchanged on the server. To remove the key on the server, use Clear() instead.

For object metadata (but not other object attributes), deleting a key will cause that key to be deleted on the server. Del() is identical to Clear() in this case.

func (Headers) Get 

func (h Headers) Get(key string) string

Get returns the value for the specified header.

func (Headers) Set 

func (h Headers) Set(key, value string)

Set sets a new value for the specified header. Any existing value will be overwritten.

func (Headers) ToHTTP 

func (h Headers) ToHTTP() http.Header

ToHTTP converts this Headers instance into the equivalent http.Header instance. The return value is guaranteed to be non-nil.

func (Headers) ToOpts 

func (h Headers) ToOpts() *RequestOptions

ToOpts wraps this Headers instance into a RequestOpts instance, so that it can be passed to Schwift's various request methods. 

hdr := NewObjectHeaders()
hdr.ContentType().Set("image/png")
hdr.Metadata().Set("color", "blue")
obj.Upload(content, nil, hdr.ToOpts())

type LargeObject 

type LargeObject struct {
	// contains filtered or unexported fields
}

LargeObject is a wrapper for type Object that performs operations specific to large objects, i.e. those objects which are uploaded in segments rather than all at once. It can be constructed with the Object.AsLargeObject() and Object.AsNewLargeObject() methods.

The following example shows how to upload a large file from the filesystem to Swift (error handling elided for brevity): 

file, err := os.Open(sourcePath)
segmentContainer, err := account.Container("segments").EnsureExists()

lo, err := o.AsNewLargeObject(schwift.SegmentingOptions {
    SegmentContainer: segmentContainer,
    //use defaults for everything else
}, &schwift.TruncateOptions {
    //if there's already a large object here, clean it up
    DeleteSegments: true,
})

err = lo.Append(contents, 1<<30) // 1<30 bytes = 1 GiB per segment
err = lo.WriteManifest(nil)

Append() has a more low-level counterpart, AddSegment(). Both methods can be freely intermixed. AddSegment() is useful when you want to control the segments' metadata or use advanced features like range segments or data segments; see documentation over there.

Writing to a large object must always be concluded by a call to WriteManifest() to link the new segments to the large object on the server side.

func (*LargeObject) AddSegment 

func (lo *LargeObject) AddSegment(segment SegmentInfo) error

AddSegment appends a segment to this object. The segment must already have been uploaded.

WARNING: This is a low-level function. Most callers will want to use Append(). You will only need to add segments manually when you want to control the segments' metadata, or when using advanced features such as range-limited segments or data segments.

This method returns ErrAccountMismatch if the segment is not located in a container in the same account.

For dynamic large objects, this method returns ErrContainerMismatch if the segment is not located in the correct container below the correct prefix.

This method returns ErrSegmentInvalid if:

- a range is specified in the SegmentInfo, but it is invalid or the LargeObject is a dynamic large object (DLOs do not support ranges), or

- the SegmentInfo's Data attribute is set and any other attribute is also set (segments cannot be backed by objects and be data segments at the same time), or

- the SegmentInfo's Data attribute is set, but the LargeObject is a dynamic large objects (DLOs do not support data segments).

func (*LargeObject) Append 

func (lo *LargeObject) Append(ctx context.Context, contents io.Reader, segmentSizeBytes int64, opts *RequestOptions) error

Append uploads the contents of the given io.Reader as segment objects of the given segment size. (The last segment will be shorter than the segment size unless the reader yields an exact multiple of the segment size.) The reader is consumed until EOF, or until an error occurs.

If you do not have an io.Reader, but you have a []byte or string instance containing the data, wrap it in a *bytes.Reader instance like so: 

var buffer []byte
lo.Append(bytes.NewReader(buffer), segmentSizeBytes)

//or...
var buffer string
lo.Append(bytes.NewReader([]byte(buffer)), segmentSizeBytes)

If segmentSizeBytes is zero, Append() defaults to the maximum file size reported by Account.Capabilities().

Calls to Append() and its low-level counterpart, AddSegment(), can be freely intermixed. AddSegment() is useful when you want to control the segments' metadata or use advanced features like range segments or data segments; see documentation over there.

This function uploads segment objects, so it may return any error that Object.Upload() returns, see documentation over there.

func (*LargeObject) NextSegmentObject 

func (lo *LargeObject) NextSegmentObject() *Object

NextSegmentObject suggests where to upload the next segment.

WARNING: This is a low-level function. Most callers will want to use Append(). You will only need to upload segments manually when you want to control the segments' metadata.

If the name of the current final segment ends with a counter, that counter is incremented, otherwise a counter is appended to its name. When looking for a counter in an existing segment name, the regex /[0-9]+$/ is used. For example, given: 

segments := lo.segments()
lastSegmentName := segments[len(segments)-1].Name()
nextSegmentName := lo.NextSegmentObject().Name()

If lastSegmentName is "segments/archive/segment0001", then nextSegmentName is "segments/archive/segment0002". If lastSegmentName is "segments/archive/first", then nextSegmentName is "segments/archive/first0000000000000001".

However, the last segment's name will only be considered if it lies within lo.segmentContainer below lo.segmentPrefix. If that is not the case, the name of the last segment that does will be used instead.

If there are no segments yet, or if all segments are located outside the lo.segmentContainer and lo.segmentPrefix, the first segment name is chosen as lo.segmentPrefix + "0000000000000001".

func (*LargeObject) Object 

func (lo *LargeObject) Object() *Object

Object returns the location of this large object (where its manifest is stored).

func (*LargeObject) SegmentContainer 

func (lo *LargeObject) SegmentContainer() *Container

SegmentContainer returns the container in which this object's segments are stored. For static large objects, some segments may also be located in different containers.

func (*LargeObject) SegmentObjects 

func (lo *LargeObject) SegmentObjects() []*Object

SegmentObjects returns a list of all segment objects referenced by this large object. Note that, in general, 

len(lo.SegmentObjects()) <= len(lo.Segments())

since one object may be backing multiple segments, and data segments are not backed by any object at all. No guarantee is made about the order in which objects appear in this list.

func (*LargeObject) SegmentPrefix 

func (lo *LargeObject) SegmentPrefix() string

SegmentPrefix returns the prefix shared by the names of all segments of this object. For static large objects, some segments may not be located in this prefix.

func (*LargeObject) Segments 

func (lo *LargeObject) Segments() ([]SegmentInfo, error)

Segments returns a list of all segments for this object, in order.

func (*LargeObject) Strategy 

func (lo *LargeObject) Strategy() LargeObjectStrategy

Strategy returns the LargeObjectStrategy used by this object.

func (*LargeObject) Truncate 

func (lo *LargeObject) Truncate(ctx context.Context, opts *TruncateOptions) error

Truncate removes all segments from a large object's manifest. The manifest is not written by this call, so WriteManifest() usually needs to be called afterwards.

func (*LargeObject) WriteManifest 

func (lo *LargeObject) WriteManifest(ctx context.Context, opts *RequestOptions) error

WriteManifest creates this large object by writing a manifest to its location using a PUT request.

For dynamic large objects, this method does not generate a PUT request if the object already exists and has the correct manifest (i.e. SegmentContainer and SegmentPrefix have not been changed).

type LargeObjectStrategy 

type LargeObjectStrategy int

LargeObjectStrategy enumerates segmenting strategies supported by Swift. 

const (
	// StaticLargeObject is the default LargeObjectStrategy used by Schwift.
	StaticLargeObject LargeObjectStrategy = iota + 1
	// DynamicLargeObject is an older LargeObjectStrategy that is not recommended
	// for new applications because of eventual consistency problems and missing
	// support for several newer features (e.g. data segments, range specifications).
	DynamicLargeObject
)

A value of 0 for LargeObjectStrategy will instruct Schwift to choose a strategy itself. Right now, Schwift always chooses StaticLargeObject, but this behavior may change in future versions of Schwift, esp. if new strategies become available. The choice may also start to depend on the capabilities advertised by the server.

type MalformedHeaderError 

type MalformedHeaderError struct {
	Key        string
	ParseError error
}

MalformedHeaderError is generated when a response from Swift contains a malformed header.

func (MalformedHeaderError) Error 

func (e MalformedHeaderError) Error() string

Error implements the builtin/error interface.

type Object 

type Object struct {
	// contains filtered or unexported fields
}

Object represents a Swift object. Instances are usually obtained by traversing downwards from a container with Container.Object() or Container.Objects().

func (*Object) AsLargeObject 

func (o *Object) AsLargeObject(ctx context.Context) (*LargeObject, error)

AsLargeObject opens an existing large object. If the given object does not exist, or if it is not a large object, ErrNotLarge will be returned. In this case, Object.AsNewLargeObject() needs to be used instead.

func (*Object) AsNewLargeObject 

func (o *Object) AsNewLargeObject(ctx context.Context, sopts SegmentingOptions, topts *TruncateOptions) (*LargeObject, error)

AsNewLargeObject opens an object as a large object. SegmentingOptions are always required, see the documentation on type SegmentingOptions for details.

This function can be used regardless of whether the object exists or not. If the object exists and is a large object, this function behaves like Object.AsLargeObject() followed by Truncate(), except that segmenting options are initialized from the method's SegmentingOptions argument rather than from the existing manifest.

func (*Object) Container 

func (o *Object) Container() *Container

Container returns a handle to the container this object is stored in.

func (*Object) CopyTo 

func (o *Object) CopyTo(ctx context.Context, target *Object, opts *CopyOptions, ropts *RequestOptions) error

CopyTo copies the object on the server side using a COPY request.

A successful COPY implies target.Invalidate() since it may change the target's metadata.

func (*Object) Delete 

func (o *Object) Delete(ctx context.Context, opts *DeleteOptions, ropts *RequestOptions) error

Delete deletes the object using a DELETE request. To add URL parameters, pass a non-nil *RequestOptions.

This operation fails with http.StatusNotFound if the object does not exist.

A successful DELETE request implies Invalidate().

func (*Object) Download 

func (o *Object) Download(ctx context.Context, opts *RequestOptions) DownloadedObject

Download retrieves the object's contents using a GET request. This returns a helper object which allows you to select whether you want an io.ReadCloser for reading the object contents progressively, or whether you want the object contents collected into a byte slice or string. 

reader, err := object.Download(nil).AsReadCloser()

buf, err := object.Download(nil).AsByteSlice()

str, err := object.Download(nil).AsString()

See documentation on type DownloadedObject for details.

WARNING: This method is not thread-safe. Calling it concurrently on the same object results in undefined behavior.

func (*Object) Exists 

func (o *Object) Exists(ctx context.Context) (bool, error)

Exists checks if this object exists, potentially by issuing a HEAD request if no Headers() have been cached yet.

func (*Object) FullName 

func (o *Object) FullName() string

FullName returns the container name and object name joined together with a slash. This identifier is used by Swift in several places (large object manifests, symlink targets, etc.) to refer to an object within an account. For example: 

obj := account.Container("docs").Object("2018-02-10/invoice.pdf")
obj.Name()     //returns      "2018-02-10/invoice.pdf"
obj.FullName() //returns "docs/2018-02-10/invoice.pdf"

func (*Object) Headers 

func (o *Object) Headers(ctx context.Context) (ObjectHeaders, error)

Headers returns the ObjectHeaders for this object. If the ObjectHeaders has not been cached yet, a HEAD request is issued on the object.

For symlinks, this operation returns the metadata for the target object. Use Object.SymlinkHeaders() to obtain the metadata for the symlink instead.

This operation fails with http.StatusNotFound if the object does not exist.

WARNING: This method is not thread-safe. Calling it concurrently on the same object results in undefined behavior.

func (*Object) Invalidate 

func (o *Object) Invalidate()

Invalidate clears the internal cache of this Object instance. The next call to Headers() on this instance will issue a HEAD request on the object.

WARNING: This method is not thread-safe. Calling it concurrently on the same object results in undefined behavior.

func (*Object) IsEqualTo 

func (o *Object) IsEqualTo(other *Object) bool

IsEqualTo returns true if both Object instances refer to the same object.

func (*Object) Name 

func (o *Object) Name() string

Name returns the object name. This does not parse the name in any way; if you want only the basename portion of the object name, use package path from the standard library in conjunction with this function. For example: 

obj := account.Container("docs").Object("2018-02-10/invoice.pdf")
obj.Name()            //returns "2018-02-10/invoice.pdf"
path.Base(obj.Name()) //returns            "invoice.pdf"

func (*Object) SymlinkHeaders 

func (o *Object) SymlinkHeaders(ctx context.Context) (headers ObjectHeaders, target *Object, err error)

SymlinkHeaders is similar to Headers, but if the object is a symlink, it returns the metadata of the symlink rather than the metadata of the target. It also returns a reference to the target object.

If this object is not a symlink, Object.SymlinkHeaders() returns the same ObjectHeaders as Object.Headers(), and a nil target object.

In a nutshell, if Object.Headers() is like os.Stat(), then Object.SymlinkHeaders() is like os.Lstat().

If you do not know whether a given object is a symlink or not, it's a good idea to call Object.SymlinkHeaders() first: If the object turns out not to be a symlink, the cache for Object.Headers() has already been populated.

This operation fails with http.StatusNotFound if the object does not exist.

WARNING: This method is not thread-safe. Calling it concurrently on the same object results in undefined behavior.

func (*Object) SymlinkTo 

func (o *Object) SymlinkTo(ctx context.Context, target *Object, opts *SymlinkOptions, ropts *RequestOptions) error

SymlinkTo creates the object as a symbolic link to another object using a PUT request. Like Object.Upload(), this method works regardless of whether the object already exists or not. Existing object contents will be overwritten by this operation.

A successful PUT request implies Invalidate() since it may change metadata.

func (*Object) TempURL 

func (o *Object) TempURL(ctx context.Context, key, method string, expires time.Time) (string, error)

TempURL is like Object.URL, but includes a token with a limited lifetime (as specified by the `expires` argument) that permits anonymous access to this object using the given HTTP method. This works only when the tempurl middleware is set up on the server, and if the given `key` matches one of the tempurl keys for this object's container or account.

For example, if the ReadACL both on the account and container do not permit anonymous read access (which is the default behavior): 

var o *schwift.Object
...
resp, err := http.Get(o.URL())
//After this, resp.StatusCode == 401 (Unauthorized)
//because anonymous access is forbidden.

//But if the container or account has a tempurl key...
key := "supersecretkey"
hdr := NewContainerHeaders()
hdr.TempURLKey().Set(key)
c := o.Container()
err := c.Update(ctx, hdr, nil)

//...we can use it to generate temporary URLs.
url := o.TempURL(ctx, key, "GET", time.Now().Add(10 * time.Minute))
resp, err := http.Get(url)
//This time, resp.StatusCode == 200 because the URL includes a token.

func (*Object) URL 

func (o *Object) URL() (string, error)

URL returns the canonical URL for the object on the server. This is particularly useful when the ReadACL on the account or container is set to allow anonymous read access.

func (*Object) Update 

func (o *Object) Update(ctx context.Context, headers ObjectHeaders, opts *RequestOptions) error

Update updates the object's headers using a POST request. To add URL parameters, pass a non-nil *RequestOptions.

This operation fails with http.StatusNotFound if the object does not exist.

A successful POST request implies Invalidate() since it may change metadata.

func (*Object) Upload 

func (o *Object) Upload(ctx context.Context, content io.Reader, opts *UploadOptions, ropts *RequestOptions) error

Upload creates the object using a PUT request.

If you do not have an io.Reader, but you have a []byte or string instance containing the object, wrap it in a *bytes.Reader instance like so: 

var buffer []byte
o.Upload(bytes.NewReader(buffer), opts)

//or...
var buffer string
o.Upload(bytes.NewReader([]byte(buffer)), opts)

If you have neither an io.Reader nor a []byte or string, but you have a function that generates the object's content into an io.Writer, use UploadFromWriter instead.

If the object is very large and you want to upload it in segments, use LargeObject.Append() instead. See documentation on type LargeObject for details.

If content is a *bytes.Reader or a *bytes.Buffer instance, the Content-Length and Etag request headers will be computed automatically. Otherwise, it is highly recommended that the caller set these headers (if possible) to allow the server to check the integrity of the uploaded file.

If Etag and/or Content-Length is supplied and the content does not match these parameters, http.StatusUnprocessableEntity is returned. If Etag is not supplied and cannot be computed in advance, Upload() will compute the Etag as data is read from the io.Reader, and compare the result to the Etag returned by Swift, returning ErrChecksumMismatch in case of mismatch. The object will have been uploaded at that point, so you will usually want to Delete() it.

This function can be used regardless of whether the object exists or not.

A successful PUT request implies Invalidate() since it may change metadata.

func (*Object) UploadFromWriter 

func (o *Object) UploadFromWriter(ctx context.Context, opts *UploadOptions, ropts *RequestOptions, callback func(io.Writer) error) error

UploadFromWriter is a variant of Upload that can be used when the object's content is generated by some function or package that takes an io.Writer instead of supplying an io.Reader. For example: 

func greeting(target io.Writer, name string) error {
    _, err := fmt.Fprintf(target, "Hello %s!\n", name)
    return err
}

obj := container.Object("greeting-for-susan-and-jeffrey")
err := obj.UploadFromWriter(nil, func(w io.Writer) error {
    err := greeting(w, "Susan")
    if err == nil {
        err = greeting(w, "Jeffrey")
    }
    return err
})

If you do not need an io.Writer, always use Upload instead.

type ObjectHeaders 

type ObjectHeaders struct {
	Headers
}

ObjectHeaders contains the headers for a schwift.Object instance.

To read and write well-known headers, use the methods on this type. To read and write arbitrary headers, use the methods on the Headers supertype.

func NewObjectHeaders 

func NewObjectHeaders() ObjectHeaders

NewObjectHeaders creates a new ObjectHeaders instance. The return value will have the Headers attribute initialized to a non-nil map.

func (ObjectHeaders) ContentDisposition 

func (h ObjectHeaders) ContentDisposition() FieldString

ContentDisposition provides type-safe access to Content-Disposition headers.

func (ObjectHeaders) ContentEncoding 

func (h ObjectHeaders) ContentEncoding() FieldString

ContentEncoding provides type-safe access to Content-Encoding headers.

func (ObjectHeaders) ContentType 

func (h ObjectHeaders) ContentType() FieldString

ContentType provides type-safe access to Content-Type headers.

func (ObjectHeaders) CreatedAt 

func (h ObjectHeaders) CreatedAt() FieldUnixTimeReadonly

CreatedAt provides type-safe access to X-Timestamp headers.

func (ObjectHeaders) Etag 

func (h ObjectHeaders) Etag() FieldString

Etag provides type-safe access to Etag headers.

func (ObjectHeaders) ExpiresAt 

func (h ObjectHeaders) ExpiresAt() FieldUnixTime

ExpiresAt provides type-safe access to X-Delete-At headers.

func (ObjectHeaders) IsDynamicLargeObject 

func (h ObjectHeaders) IsDynamicLargeObject() bool

IsDynamicLargeObject returns true if this set of headers belongs to a Dynamic Large Object (DLO).

func (ObjectHeaders) IsLargeObject 

func (h ObjectHeaders) IsLargeObject() bool

IsLargeObject returns true if this set of headers belongs to a large object (either an SLO or a DLO).

func (ObjectHeaders) IsStaticLargeObject 

func (h ObjectHeaders) IsStaticLargeObject() bool

IsStaticLargeObject returns true if this set of headers belongs to a Static Large Object (SLO).

func (ObjectHeaders) Metadata 

func (h ObjectHeaders) Metadata() FieldMetadata

Metadata provides type-safe access to X-Object-Meta- headers.

func (ObjectHeaders) SizeBytes 

func (h ObjectHeaders) SizeBytes() FieldUint64

SizeBytes provides type-safe access to Content-Length headers.

func (ObjectHeaders) SymlinkTarget 

func (h ObjectHeaders) SymlinkTarget() FieldString

SymlinkTarget provides type-safe access to X-Symlink-Target headers.

func (ObjectHeaders) SymlinkTargetAccount 

func (h ObjectHeaders) SymlinkTargetAccount() FieldString

SymlinkTargetAccount provides type-safe access to X-Symlink-Target-Account headers.

func (ObjectHeaders) UpdatedAt 

func (h ObjectHeaders) UpdatedAt() FieldHTTPTimeReadonly

UpdatedAt provides type-safe access to Last-Modified headers.

func (ObjectHeaders) Validate 

func (h ObjectHeaders) Validate() error

Validate returns MalformedHeaderError if the value of any well-known header does not conform to its data type. This is called automatically by Schwift when preparing an ObjectHeaders instance from a GET/HEAD response, so you usually do not need to do it yourself. You will get the validation error from the Object method doing the request, e.g. Headers().

type ObjectInfo 

type ObjectInfo struct {
	Object       *Object
	SizeBytes    uint64
	ContentType  string
	Etag         string
	LastModified time.Time
	// SymlinkTarget is only set for symlinks.
	SymlinkTarget *Object
	// If the ObjectInfo refers to an actual object, then SubDirectory is empty.
	// If the ObjectInfo refers to a pseudo-directory, then SubDirectory contains
	// the path of the pseudo-directory and all other fields are nil/zero/empty.
	// Pseudo-directories will only be reported for ObjectIterator.Delimiter != "".
	SubDirectory string
}

ObjectInfo is a result type returned by ObjectIterator for detailed object listings. The metadata in this type is a subset of Object.Headers(), but since it is returned as part of the detailed object listing, it can be obtained without making additional HEAD requests on the object(s).

type ObjectIterator 

type ObjectIterator struct {
	Container *Container
	// When Prefix is set, only objects whose name starts with this string are
	// returned.
	Prefix string
	// When Delimiter is set, objects whose name contains this string (after the
	// prefix, if any) will be condensed into pseudo-directories in the result.
	// See documentation for Swift for details.
	Delimiter string
	// Options may contain additional headers and query parameters for the GET request.
	Options *RequestOptions
	// contains filtered or unexported fields
}

ObjectIterator iterates over the objects in a container. It is typically constructed with the Container.Objects() method. For example: 

//either this...
iter := container.Objects()
iter.Prefix = "test-"
objects, err := iter.Collect()

//...or this
objects, err := schwift.ObjectIterator{
	Container: container,
	Prefix: "test-",
}.Collect()

When listing objects via a GET request on the container, you can choose to receive object names only (via the methods without the "Detailed" suffix), or object names plus some basic metadata fields (via the methods with the "Detailed" suffix). See struct ObjectInfo for which metadata is returned.

To obtain any other metadata, you can call Object.Headers() on the result object, but this will issue a separate HEAD request for each object.

Use the "Detailed" methods only when you use the extra metadata in struct ObjectInfo; detailed GET requests are more expensive than simple ones that return only object names.

Note that, when Delimiter is set, instances of *Object that you receive from the iterator may refer to a pseudo-directory instead of an actual object, in which case Exists() will return false.

func (*ObjectIterator) Collect 

func (i *ObjectIterator) Collect(ctx context.Context) ([]*Object, error)

Collect lists all object names matching this iterator. For large sets of objects that cannot be retrieved at once, Collect handles paging behind the scenes. The return value is always the complete set of objects.

func (*ObjectIterator) CollectDetailed 

func (i *ObjectIterator) CollectDetailed(ctx context.Context) ([]ObjectInfo, error)

CollectDetailed is like Collect, but includes basic metadata.

func (*ObjectIterator) Foreach 

func (i *ObjectIterator) Foreach(ctx context.Context, callback func(*Object) error) error

Foreach lists the object names matching this iterator and calls the callback once for every object. Iteration is aborted when a GET request fails, or when the callback returns a non-nil error.

func (*ObjectIterator) ForeachDetailed 

func (i *ObjectIterator) ForeachDetailed(ctx context.Context, callback func(ObjectInfo) error) error

ForeachDetailed is like Foreach, but includes basic metadata.

func (*ObjectIterator) NextPage 

func (i *ObjectIterator) NextPage(ctx context.Context, limit int) ([]*Object, error)

NextPage queries Swift for the next page of object names. If limit is >= 0, not more than that many object names will be returned at once. Note that the server also has a limit for how many objects to list in one request; the lower limit wins.

The end of the object listing is reached when an empty list is returned.

This method offers maximal flexibility, but most users will prefer the simpler interfaces offered by Collect() and Foreach().

func (*ObjectIterator) NextPageDetailed 

func (i *ObjectIterator) NextPageDetailed(ctx context.Context, limit int) ([]ObjectInfo, error)

NextPageDetailed is like NextPage, but includes basic metadata.

type Request 

type Request struct {
	Method        string // "GET", "HEAD", "PUT", "POST" or "DELETE"
	ContainerName string // empty for requests on accounts
	ObjectName    string // empty for requests on accounts/containers
	Options       *RequestOptions
	Body          io.Reader
	// ExpectStatusCodes can be left empty to disable this check, otherwise
	// schwift.UnexpectedStatusCodeError may be returned.
	ExpectStatusCodes []int
	// DrainResponseBody can be set if the caller is not interested in the
	// response body. This is implied for Response.StatusCode == 204.
	DrainResponseBody bool
}

Request contains the parameters that can be set in a request to the Swift API.

func (Request) Do 

func (r Request) Do(ctx context.Context, backend Backend) (*http.Response, error)

Do executes this request on the given Backend.

func (Request) URL 

func (r Request) URL(backend Backend, values url.Values) (string, error)

URL returns the full URL for this request.

type RequestOptions 

type RequestOptions struct {
	Headers Headers
	Values  url.Values
}

RequestOptions is used to pass additional headers and values to a request.

When preparing a RequestOptions instance with additional headers, the preferred way is to create an AccountHeaders, ContainerHeaders and ObjectHeaders instance and use the type-safe API on these types. Then use the ToOpts() method on that instance. For example: 

hdr := NewObjectHeaders()
hdr.ContentType().Set("image/png")
hdr.Metadata().Set("color", "blue")
opts := hdr.ToOpts() //type *schwift.RequestOptions

type SegmentInfo 

type SegmentInfo struct {
	Object      *Object
	SizeBytes   uint64
	Etag        string
	RangeLength uint64
	RangeOffset int64
	// Static Large Objects support data segments that are not backed by actual
	// objects. For those kinds of segments, only the Data attribute is set and
	// all other attributes are set to their default values (esp. .Object == nil).
	//
	// Data segments can only be used for small chunks of data because the SLO
	// manifest (the list of all SegmentInfo encoded as JSON) is severely limited
	// in size (usually to 8 MiB).
	Data []byte
}

SegmentInfo describes a segment of a large object.

For .RangeLength == 0, the segment consists of all the bytes in the backing object, after skipping the first .RangeOffset bytes. The default (.RangeOffset == 0) is to include the entire contents of the backing object.

For .RangeLength > 0, the segment consists of that many bytes from the backing object, again after skipping the first .RangeOffset bytes.

However, for .RangeOffset < 0, the segment consists of .RangeLength many bytes from the *end* of the backing object. (The concrete value for .RangeOffset is disregarded.) .RangeLength must be non-zero in this case.

Sorry that specifying a range is that involved. I was just following orders ^W RFC 7233, section 3.1 here.

type SegmentingOptions 

type SegmentingOptions struct {
	Strategy         LargeObjectStrategy
	SegmentContainer *Container
	SegmentPrefix    string
}

SegmentingOptions describes how an object is segmented. It is passed to Object.AsNewLargeObject().

If Strategy is not set, a reasonable strategy is chosen; see documentation on LargeObjectStrategy for details.

SegmentContainer must not be nil. A value of nil will cause Schwift to panic. If the SegmentContainer is not in the same account as the large object, ErrAccountMismatch will be returned by Schwift.

If SegmentPrefix is empty, a reasonable default will be computed by Object.AsNewLargeObject(), using the format "<object-name>/<strategy>/<timestamp>", where strategy is either "slo" or "dlo".

type StoragePolicySpec 

type StoragePolicySpec struct {
	Name    string `json:"name"`
	Aliases string `json:"aliases"`
	Default bool   `json:"default"`
}

StoragePolicySpec is a subtype that appears in struct Capabilities.

type SymlinkOptions 

type SymlinkOptions struct {
	// When overwriting a large object, delete its segments. This will cause
	// SymlinkTo() to call into BulkDelete(), so a BulkError may be returned.
	DeleteSegments bool
}

SymlinkOptions invokes advanced behavior in the Object.SymlinkTo() method.

type TruncateOptions 

type TruncateOptions struct {
	// When truncating a large object's manifest, delete its segments.
	// This will cause Truncate() to call into BulkDelete(), so a BulkError may be
	// returned. If this is false, the segments will not be deleted even though
	// they may not be referenced by any large object anymore.
	DeleteSegments bool
}

TruncateOptions contains options that can be passed to LargeObject.Truncate() and Object.AsNewLargeObject().

type UnexpectedStatusCodeError 

type UnexpectedStatusCodeError struct {
	Method              string // e.g. http.MethodGet
	Target              string // either "<account>" or "$CONTAINER_NAME" or "$CONTAINER_NAME/$OBJECT_NAME"
	ExpectedStatusCodes []int
	ActualResponse      *http.Response
	ResponseBody        []byte
}

UnexpectedStatusCodeError is generated when a request to Swift does not yield a response with the expected successful status code. The actual status code can be checked with the Is() function; see documentation over there.

func (UnexpectedStatusCodeError) Error 

func (e UnexpectedStatusCodeError) Error() string

Error implements the builtin/error interface.

type UploadOptions 

type UploadOptions struct {
	// When overwriting a large object, delete its segments. This will cause
	// Upload() to call into BulkDelete(), so a BulkError may be returned.
	DeleteSegments bool
}

UploadOptions invokes advanced behavior in the Object.Upload() method.

Source Files

View all Source files

Directories

Path Synopsis
Package capabilities contains feature switches that Schwift's unit tests can set to exercise certain fallback code paths in Schwift that they could not trigger otherwise.
 Package capabilities contains feature switches that Schwift's unit tests can set to exercise certain fallback code paths in Schwift that they could not trigger otherwise.
Package gopherschwift contains a Gophercloud backend for Schwift.
 Package gopherschwift contains a Gophercloud backend for Schwift.
internal
errext

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.