genai

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 16, 2025 License: Apache-2.0 Imports: 31 Imported by: 55

README

GitHub go.mod Go version Go Reference

✨ NEW ✨

Google Gemini Multimodal Live support

Introducing support for the Gemini Multimodal Live feature. Here's an example Multimodal Live server showing realtime conversation and video streaming: code

Google Gen AI Go SDK

The Google Gen AI Go SDK enables developers to use Google's state-of-the-art generative AI models (like Gemini) to build AI-powered features and applications. This SDK supports use cases like:

  • Generate text from text-only input
  • Generate text from text-and-images input (multimodal)
  • ...

For example, with just a few lines of code, you can access Gemini's multimodal capabilities to generate text from text-and-image input.

parts := []*genai.Part{
  {Text: "What's this image about?"},
  {InlineData: &genai.Blob{Data: imageBytes, MIMEType: "image/jpeg"}},
}
result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", []*genai.Content{{Parts: parts}}, nil)

Installation and usage

Add the SDK to your module with go get google.golang.org/genai.

Create Clients

Imports

import "google.golang.org/genai"

Gemini API Client:

client, err := genai.NewClient(ctx, &genai.ClientConfig{
	APIKey:   apiKey,
	Backend:  genai.BackendGeminiAPI,
})

Vertex AI Client:

client, err := genai.NewClient(ctx, &genai.ClientConfig{
	Project:  project,
	Location: location,
	Backend:  genai.BackendVertexAI,
})

License

The contents of this repository are licensed under the Apache License, version 2.0.

Documentation

Index

Examples

Constants

View Source
const (
	RoleUser  = "user"
	RoleModel = "model"
)

Variables

View Source
var ErrPageDone = errors.New("no more pages")

ErrPageDone is the error returned by an iterator's Next method when no more pages are available.

Functions

func Ptr

func Ptr[T any](t T) *T

Ptr returns a pointer to its argument. It can be used to initialize pointer fields:

genai.GenerateContentConfig{Temperature: genai.Ptr(0.5)}

Types

type APIError added in v0.7.0

type APIError struct {
	// Code is the HTTP response status code.
	Code int `json:"code,omitempty"`
	// Message is the server response message.
	Message string `json:"message,omitempty"`
	// Status is the server response status.
	Status string `json:"status,omitempty"`
	// Details field provides more context to an error.
	Details []map[string]any `json:"details,omitempty"`
}

APIError contains an error response from the server.

func (APIError) Error added in v0.7.0

func (e APIError) Error() string

Error returns a string representation of the APIError.

type ActivityEnd added in v1.0.0

type ActivityEnd struct {
}

Marks the end of user activity. This can only be sent if automatic (i.e. server-side) activity detection is disabled.

type ActivityHandling added in v1.0.0

type ActivityHandling string

The different ways of handling user activity.

const (
	// If unspecified, the default behavior is `START_OF_ACTIVITY_INTERRUPTS`.
	ActivityHandlingUnspecified ActivityHandling = "ACTIVITY_HANDLING_UNSPECIFIED"
	// If true, start of activity will interrupt the model's response (also called "barge
	// in"). The model's current response will be cut-off in the moment of the interruption.
	// This is the default behavior.
	ActivityHandlingStartOfActivityInterrupts ActivityHandling = "START_OF_ACTIVITY_INTERRUPTS"
	// The model's response will not be interrupted.
	ActivityHandlingNoInterruption ActivityHandling = "NO_INTERRUPTION"
)

type ActivityStart added in v1.0.0

type ActivityStart struct {
}

Marks the start of user activity. This can only be sent if automatic (i.e. server-side) activity detection is disabled.

type AudioTranscriptionConfig added in v1.0.0

type AudioTranscriptionConfig struct {
}

The audio transcription configuration in Setup.

type AutomaticActivityDetection added in v1.0.0

type AutomaticActivityDetection struct {
	// If enabled, detected voice and text input count as activity. If disabled, the client
	// must send activity signals.
	Disabled bool `json:"disabled,omitempty"`
	// Determines how likely speech is to be detected.
	StartOfSpeechSensitivity StartSensitivity `json:"startOfSpeechSensitivity,omitempty"`
	// Determines how likely detected speech is ended.
	EndOfSpeechSensitivity EndSensitivity `json:"endOfSpeechSensitivity,omitempty"`
	// The required duration of detected speech before start-of-speech is committed. The
	// lower this value the more sensitive the start-of-speech detection is and the shorter
	// speech can be recognized. However, this also increases the probability of false positives.
	PrefixPaddingMs *int32 `json:"prefixPaddingMs,omitempty"`
	// The required duration of detected non-speech (e.g. silence) before end-of-speech
	// is committed. The larger this value, the longer speech gaps can be without interrupting
	// the user's activity but this will increase the model's latency.
	SilenceDurationMs *int32 `json:"silenceDurationMs,omitempty"`
}

Configures automatic detection of activity.

type Backend

type Backend int

Backend is the GenAI backend to use for the client.

const (
	// BackendUnspecified causes the backend determined automatically. If the
	// GOOGLE_GENAI_USE_VERTEXAI environment variable is set to "1" or "true", then
	// the backend is BackendVertexAI. Otherwise, if GOOGLE_GENAI_USE_VERTEXAI
	// is unset or set to any other value, then BackendGeminiAPI is used.  Explicitly
	// setting the backend in ClientConfig overrides the environment variable.
	BackendUnspecified Backend = iota
	// BackendGeminiAPI is the Gemini API backend.
	BackendGeminiAPI
	// BackendVertexAI is the Vertex AI backend.
	BackendVertexAI
)

func (Backend) String

func (t Backend) String() string

The Stringer interface for Backend.

type Blob

type Blob struct {
	// Required. Raw bytes.
	Data []byte `json:"data,omitempty"`
	// Required. The IANA standard MIME type of the source data.
	MIMEType string `json:"mimeType,omitempty"`
}

Content blob.

type BlockedReason

type BlockedReason string

Blocked reason.

const (
	// Unspecified blocked reason.
	BlockedReasonUnspecified BlockedReason = "BLOCKED_REASON_UNSPECIFIED"
	// Candidates blocked due to safety.
	BlockedReasonSafety BlockedReason = "SAFETY"
	// Candidates blocked due to other reason.
	BlockedReasonOther BlockedReason = "OTHER"
	// Candidates blocked due to the terms which are included from the terminology blocklist.
	BlockedReasonBlocklist BlockedReason = "BLOCKLIST"
	// Candidates blocked due to prohibited content.
	BlockedReasonProhibitedContent BlockedReason = "PROHIBITED_CONTENT"
)

type CachedContent added in v0.1.0

type CachedContent struct {
	// The server-generated resource name of the cached content.
	Name string `json:"name,omitempty"`
	// The user-generated meaningful display name of the cached content.
	DisplayName string `json:"displayName,omitempty"`
	// The name of the publisher model to use for cached content.
	Model string `json:"model,omitempty"`
	// Creation time of the cache entry.
	CreateTime time.Time `json:"createTime,omitempty"`
	// When the cache entry was last updated in UTC time.
	UpdateTime time.Time `json:"updateTime,omitempty"`
	// Expiration time of the cached content.
	ExpireTime time.Time `json:"expireTime,omitempty"`
	// Metadata on the usage of the cached content.
	UsageMetadata *CachedContentUsageMetadata `json:"usageMetadata,omitempty"`
}

A resource used in LLM queries for users to explicitly specify what to cache.

func (*CachedContent) MarshalJSON added in v0.6.0

func (c *CachedContent) MarshalJSON() ([]byte, error)

type CachedContentUsageMetadata added in v0.1.0

type CachedContentUsageMetadata struct {
	// Duration of audio in seconds.
	AudioDurationSeconds int32 `json:"audioDurationSeconds,omitempty"`
	// Number of images.
	ImageCount int32 `json:"imageCount,omitempty"`
	// Number of text characters.
	TextCount int32 `json:"textCount,omitempty"`
	// Total number of tokens that the cached content consumes.
	TotalTokenCount int32 `json:"totalTokenCount,omitempty"`
	// Duration of video in seconds.
	VideoDurationSeconds int32 `json:"videoDurationSeconds,omitempty"`
}

Metadata on the usage of the cached content.

type Caches added in v0.1.0

type Caches struct {
	// contains filtered or unexported fields
}

Caches provides methods for managing the context caching. You don't need to initiate this struct. Create a client instance via NewClient, and then access Caches through client.Caches field.

func (Caches) All added in v0.4.0

All retrieves all cached content resources.

This method handles pagination internally, making multiple API calls as needed to fetch all entries. It returns an iterator that yields each cached content entry one by one. You do not need to manage pagination tokens or make multiple calls to retrieve all data.

func (Caches) Create added in v0.1.0

func (m Caches) Create(ctx context.Context, model string, config *CreateCachedContentConfig) (*CachedContent, error)

Create creates a new cached content resource.

func (Caches) Delete added in v0.1.0

Delete deletes a cached content resource.

func (Caches) Get added in v0.1.0

func (m Caches) Get(ctx context.Context, name string, config *GetCachedContentConfig) (*CachedContent, error)

Get gets a cached content resource.

func (Caches) List added in v0.4.0

List retrieves a paginated list of cached content resources.

func (Caches) Update added in v0.1.0

func (m Caches) Update(ctx context.Context, name string, config *UpdateCachedContentConfig) (*CachedContent, error)

Update updates a cached content resource.

type Candidate

type Candidate struct {
	// Contains the multi-part content of the response.
	Content *Content `json:"content,omitempty"`
	// Source attribution of the generated content.
	CitationMetadata *CitationMetadata `json:"citationMetadata,omitempty"`
	// Describes the reason the model stopped generating tokens.
	FinishMessage string `json:"finishMessage,omitempty"`
	// Number of tokens for this candidate.
	// This field is only available in the Gemini API.
	TokenCount int32 `json:"tokenCount,omitempty"`
	// The reason why the model stopped generating tokens.
	// If empty, the model has not stopped generating the tokens.
	FinishReason FinishReason `json:"finishReason,omitempty"`
	// Output only. Average log probability score of the candidate.
	AvgLogprobs float64 `json:"avgLogprobs,omitempty"`
	// Output only. Metadata specifies sources used to ground generated content.
	GroundingMetadata *GroundingMetadata `json:"groundingMetadata,omitempty"`
	// Output only. Index of the candidate.
	Index int32 `json:"index,omitempty"`
	// Output only. Log-likelihood scores for the response tokens and top tokens
	LogprobsResult *LogprobsResult `json:"logprobsResult,omitempty"`
	// Output only. List of ratings for the safety of a response candidate. There is at
	// most one rating per category.
	SafetyRatings []*SafetyRating `json:"safetyRatings,omitempty"`
}

A response candidate generated from the model.

type Chat added in v0.7.0

type Chat struct {
	Models
	// contains filtered or unexported fields
}

Chat represents a single chat session (multi-turn conversation) with the model.

	client, _ := genai.NewClient(ctx, &genai.ClientConfig{})
	chat, _ := client.Chats.Create(ctx, "gemini-2.0-flash", nil, nil)
  result, err = chat.SendMessage(ctx, genai.Part{Text: "What is 1 + 2?"})

func (*Chat) History added in v1.0.0

func (c *Chat) History(curated bool) []*Content

History returns the chat history. Curated (valid only) history is not supported yet.

func (*Chat) SendMessage added in v0.7.0

func (c *Chat) SendMessage(ctx context.Context, parts ...Part) (*GenerateContentResponse, error)

SendMessage sends the conversation history with the additional user's message and returns the model's response.

func (*Chat) SendMessageStream added in v1.0.0

func (c *Chat) SendMessageStream(ctx context.Context, parts ...Part) iter.Seq2[*GenerateContentResponse, error]

SendMessageStream sends the conversation history with the additional user's message and returns the model's response.

type Chats added in v0.7.0

type Chats struct {
	// contains filtered or unexported fields
}

Chats provides util functions for creating a new chat session. You don't need to initiate this struct. Create a client instance via NewClient, and then access Chats through client.Models field.

Example (Geminiapi)
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	chat, err := client.Chats.Create(ctx, "gemini-2.0-flash", nil, nil)
	if err != nil {
		log.Fatal(err)
	}

	result, err := chat.SendMessage(ctx, genai.Part{Text: "What's the weather in New York?"})
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)

	result, err = chat.SendMessage(ctx, genai.Part{Text: "How about San Francisco?"})
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (Stream_geminiapi)
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	chat, err := client.Chats.Create(ctx, "gemini-2.0-flash", nil, nil)
	if err != nil {
		log.Fatal(err)
	}

	for result, err := range chat.SendMessageStream(ctx, genai.Part{Text: "What's the weather in New York?"}) {
		if err != nil {
			log.Fatal(err)
		}
		debugPrint(result)
	}

	for result, err := range chat.SendMessageStream(ctx, genai.Part{Text: "How about San Francisco?"}) {
		if err != nil {
			log.Fatal(err)
		}
		debugPrint(result)
	}
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (Stream_vertexai)
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	chat, err := client.Chats.Create(ctx, "gemini-2.0-flash", nil, nil)
	if err != nil {
		log.Fatal(err)
	}

	for result, err := range chat.SendMessageStream(ctx, genai.Part{Text: "What's the weather in New York?"}) {
		if err != nil {
			log.Fatal(err)
		}
		debugPrint(result)
	}

	for result, err := range chat.SendMessageStream(ctx, genai.Part{Text: "How about San Francisco?"}) {
		if err != nil {
			log.Fatal(err)
		}
		debugPrint(result)
	}
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (Vertexai)
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	chat, err := client.Chats.Create(ctx, "gemini-2.0-flash", nil, nil)
	if err != nil {
		log.Fatal(err)
	}

	result, err := chat.SendMessage(ctx, genai.Part{Text: "What's the weather in New York?"})
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)

	result, err = chat.SendMessage(ctx, genai.Part{Text: "How about San Francisco?"})
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

func (*Chats) Create added in v0.7.0

func (c *Chats) Create(ctx context.Context, model string, config *GenerateContentConfig, history []*Content) (*Chat, error)

Create initializes a new chat session.

type Citation

type Citation struct {
	// Output only. End index into the content.
	EndIndex int32 `json:"endIndex,omitempty"`
	// Output only. License of the attribution.
	License string `json:"license,omitempty"`
	// Output only. Publication date of the attribution.
	PublicationDate civil.Date `json:"publicationDate,omitempty"`
	// Output only. Start index into the content.
	StartIndex int32 `json:"startIndex,omitempty"`
	// Output only. Title of the attribution.
	Title string `json:"title,omitempty"`
	// Output only. URL reference of the attribution.
	URI string `json:"uri,omitempty"`
}

Source attributions for content.

func (*Citation) MarshalJSON added in v0.6.0

func (c *Citation) MarshalJSON() ([]byte, error)

func (*Citation) UnmarshalJSON added in v0.4.0

func (c *Citation) UnmarshalJSON(data []byte) error

UnmarshalJSON custom unmarshalling to handle PublicationDate as a map containing year, month, and day.

type CitationMetadata

type CitationMetadata struct {
	// Contains citation information when the model directly quotes, at
	// length, from another source. Can include traditional websites and code
	// repositories.
	Citations []*Citation `json:"citations,omitempty"`
}

Citation information when the model quotes another source.

type Client

type Client struct {

	// Models provides access to the Models service.
	Models *Models
	// Live provides access to the Live service.
	Live *Live
	// Caches provides access to the Caches service.
	Caches *Caches
	// Chats provides util functions for creating a new chat session.
	Chats *Chats
	// Files provides access to the Files service.
	Files *Files
	// Operations provides access to long-running operations.
	Operations *Operations
	// contains filtered or unexported fields
}

Client is the GenAI client. It provides access to the various GenAI services.

func NewClient

func NewClient(ctx context.Context, cc *ClientConfig) (*Client, error)

NewClient creates a new GenAI client.

You can configure the client by passing in a ClientConfig struct.

If a nil ClientConfig is provided, the client will be configured using default settings and environment variables:

  • Environment Variables for BackendGeminiAPI:

  • GOOGLE_API_KEY: Required. Specifies the API key for the Gemini API.

  • Environment Variables for BackendVertexAI:

  • GOOGLE_GENAI_USE_VERTEXAI: Must be set to "1" or "true" to use the Vertex AI backend.

  • GOOGLE_CLOUD_PROJECT: Required. Specifies the GCP project ID.

  • GOOGLE_CLOUD_LOCATION or GOOGLE_CLOUD_REGION: Required. Specifies the GCP location/region.

If using the Vertex AI backend and no credentials are provided in the ClientConfig, the client will attempt to use application default credentials.

Example (Geminiapi)

This example shows how to create a new client for Gemini API.

package main

import (
	"context"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatalf("failed to create client: %v", err)
	}
	fmt.Println(client.ClientConfig().APIKey)
}
Output:

Example (Vertexai)

This example shows how to create a new client for Vertex AI.

package main

import (
	"context"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatalf("failed to create client: %v", err)
	}
	fmt.Println(client.ClientConfig().Backend)
}
Output:

func (Client) ClientConfig

func (c Client) ClientConfig() ClientConfig

ClientConfig returns the ClientConfig for the client.

The returned ClientConfig is a copy of the ClientConfig used to create the client.

type ClientConfig

type ClientConfig struct {
	// API Key for GenAI. Required for BackendGeminiAPI. Can also be set via the GOOGLE_API_KEY environment variable.
	APIKey string

	// Backend for GenAI. See Backend constants. Defaults to BackendGeminiAPI unless explicitly set to BackendVertexAI,
	// or the environment variable GOOGLE_GENAI_USE_VERTEXAI is set to "1" or "true".
	Backend Backend

	// GCP Project ID for Vertex AI. Required for BackendVertexAI. Can also be set via the GOOGLE_CLOUD_PROJECT environment variable.
	Project string

	// GCP Location/Region for Vertex AI. Required for BackendVertexAI. See https://cloud.google.com/vertex-ai/docs/general/locations.
	// Can also be set via the GOOGLE_CLOUD_LOCATION or GOOGLE_CLOUD_REGION environment variable.
	Location string

	// Optional. Google credentials.  If not specified, [Application Default Credentials] will be used.
	//
	// [Application Default Credentials]: https://developers.google.com/accounts/docs/application-default-credentials
	Credentials *auth.Credentials

	// Optional HTTP client to use. If nil, a default client will be created.
	// For Vertex AI, this client must handle authentication appropriately.
	HTTPClient *http.Client

	// Optional HTTP options to override.
	HTTPOptions HTTPOptions
	// contains filtered or unexported fields
}

ClientConfig is the configuration for the GenAI client.

type CodeExecutionResult

type CodeExecutionResult struct {
	// Required. Outcome of the code execution.
	Outcome Outcome `json:"outcome,omitempty"`
	// Optional. Contains stdout when code execution is successful, stderr or other description
	// otherwise.
	Output string `json:"output,omitempty"`
}

Result of executing the ExecutableCode. Always follows a `part` containing the ExecutableCode.

type ComputeTokensConfig added in v0.1.0

type ComputeTokensConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

Optional parameters for computing tokens.

type ComputeTokensResponse added in v0.1.0

type ComputeTokensResponse struct {
	// Lists of tokens info from the input. A ComputeTokensRequest could have multiple instances
	// with a prompt in each instance. We also need to return lists of tokens info for the
	// request with multiple instances.
	TokensInfo []*TokensInfo `json:"tokensInfo,omitempty"`
}

Response for computing tokens.

type Content

type Content struct {
	// List of parts that constitute a single message. Each part may have
	// a different IANA MIME type.
	Parts []*Part `json:"parts,omitempty"`
	// Optional. The producer of the content. Must be either 'user' or
	// 'model'. Useful to set for multi-turn conversations, otherwise can be
	// empty. If role is not specified, SDK will determine the role.
	Role string `json:"role,omitempty"`
}

Contains the multi-part content of a message.

func NewContentFromBytes added in v0.7.0

func NewContentFromBytes(data []byte, mimeType string, role Role) *Content

NewContentFromBytes builds a Content from a byte slice and mime type. If role is the empty string, it defaults to RoleUser.

func NewContentFromCodeExecutionResult added in v0.7.0

func NewContentFromCodeExecutionResult(outcome Outcome, output string, role Role) *Content

NewContentFromCodeExecutionResult builds a Content from a given Outcome and std output of the code execution. If role is the empty string, it defaults to RoleUser.

func NewContentFromExecutableCode added in v0.7.0

func NewContentFromExecutableCode(code string, language Language, role Role) *Content

NewContentFromExecutableCode builds a Content from a single piece of source code in the given Language. If role is the empty string, it defaults to RoleUser.

func NewContentFromFunctionCall added in v0.7.0

func NewContentFromFunctionCall(name string, args map[string]any, role Role) *Content

NewContentFromFunctionCall builds a Content from a single FunctionCall given the function name and args. If role is the empty string, it defaults to RoleUser.

func NewContentFromFunctionResponse added in v0.7.0

func NewContentFromFunctionResponse(name string, response map[string]any, role Role) *Content

NewContentFromFunctionResponse builds a Content from a single FunctionResponse given the function name and response. If role is the empty string, it defaults to RoleUser.

func NewContentFromParts added in v0.7.0

func NewContentFromParts(parts []*Part, role Role) *Content

NewContentFromParts builds a Content from a list of parts and a Role. If role is the empty string, it defaults to RoleUser.

func NewContentFromText added in v0.7.0

func NewContentFromText(text string, role Role) *Content

NewContentFromText builds a Content from a text string. If role is the empty string, it defaults to RoleUser.

func NewContentFromURI added in v0.7.0

func NewContentFromURI(fileURI, mimeType string, role Role) *Content

NewContentFromURI builds a Content from a file URI and mime type. If role is the empty string, it defaults to RoleUser.

func Text

func Text(text string) []*Content

Text returns a slice of Content with a single Part with the given text.

type ContentEmbedding added in v0.5.0

type ContentEmbedding struct {
	// A list of floats representing an embedding.
	Values []float32 `json:"values,omitempty"`
	// Vertex API only. Statistics of the input text associated with this
	// embedding.
	Statistics *ContentEmbeddingStatistics `json:"statistics,omitempty"`
}

The embedding generated from an input content.

type ContentEmbeddingStatistics added in v0.5.0

type ContentEmbeddingStatistics struct {
	// Vertex API only. If the input text was truncated due to having
	// a length longer than the allowed maximum input.
	Truncated bool `json:"truncated,omitempty"`
	// Vertex API only. Number of tokens of the input text.
	TokenCount float32 `json:"tokenCount,omitempty"`
}

Statistics of the input text associated with the result of content embedding.

type ContextWindowCompressionConfig added in v1.0.0

type ContextWindowCompressionConfig struct {
	// Number of tokens (before running turn) that triggers context window compression mechanism.
	TriggerTokens *int64 `json:"triggerTokens,omitempty"`
	// Sliding window compression mechanism.
	SlidingWindow *SlidingWindow `json:"slidingWindow,omitempty"`
}

Enables context window compression -- mechanism managing model context window so it does not exceed given length.

func (*ContextWindowCompressionConfig) MarshalJSON added in v1.0.0

func (c *ContextWindowCompressionConfig) MarshalJSON() ([]byte, error)

func (*ContextWindowCompressionConfig) UnmarshalJSON added in v1.0.0

func (c *ContextWindowCompressionConfig) UnmarshalJSON(data []byte) error

type ControlReferenceConfig

type ControlReferenceConfig struct {
	// The type of control reference image to use.
	ControlType ControlReferenceType `json:"controlType,omitempty"`
	// Defaults to False. When set to True, the control image will be
	// computed by the model based on the control type. When set to False,
	// the control image must be provided by the user.
	EnableControlImageComputation bool `json:"enableControlImageComputation,omitempty"`
}

Configuration for a Control reference image.

type ControlReferenceImage

type ControlReferenceImage struct {
	// The reference image for the editing operation.
	ReferenceImage *Image `json:"referenceImage,omitempty"`
	// The ID of the reference image.
	ReferenceID int32 `json:"referenceId,omitempty"`

	// Configuration for the control reference image.
	Config *ControlReferenceConfig `json:"config,omitempty"`
	// contains filtered or unexported fields
}

A control image is an image that represents a sketch image of areas for the model to fill in based on the prompt. Its image is either a control image provided by the user, or a regular image which the backend will use to generate a control image of. In the case of the latter, the EnableControlImageComputation field in the config should be set to true.

func NewControlReferenceImage added in v0.5.0

func NewControlReferenceImage(referenceImage *Image, referenceID int32, config *ControlReferenceConfig) *ControlReferenceImage

NewControlReferenceImage creates a new ControlReferenceImage.

type ControlReferenceType

type ControlReferenceType string

Enum representing the control type of a control reference image.

const (
	ControlReferenceTypeDefault  ControlReferenceType = "CONTROL_TYPE_DEFAULT"
	ControlReferenceTypeCanny    ControlReferenceType = "CONTROL_TYPE_CANNY"
	ControlReferenceTypeScribble ControlReferenceType = "CONTROL_TYPE_SCRIBBLE"
	ControlReferenceTypeFaceMesh ControlReferenceType = "CONTROL_TYPE_FACE_MESH"
)

type CountTokensConfig added in v0.1.0

type CountTokensConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// Instructions for the model to steer it toward better performance.
	SystemInstruction *Content `json:"systemInstruction,omitempty"`
	// Code that enables the system to interact with external systems to
	// perform an action outside of the knowledge and scope of the model.
	Tools []*Tool `json:"tools,omitempty"`
	// Configuration that the model uses to generate the response. Not
	// supported by the Gemini Developer API.
	GenerationConfig *GenerationConfig `json:"generationConfig,omitempty"`
}

Config for the count_tokens method.

type CountTokensResponse added in v0.1.0

type CountTokensResponse struct {
	// Total number of tokens.
	TotalTokens int32 `json:"totalTokens,omitempty"`
	// Number of tokens in the cached part of the prompt (the cached content). This field
	// is only available in the Gemini API.
	CachedContentTokenCount int32 `json:"cachedContentTokenCount,omitempty"`
}

Response for counting tokens.

type CreateCachedContentConfig added in v0.1.0

type CreateCachedContentConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// The TTL for this resource. The expiration time is computed: now + TTL.
	TTL time.Duration `json:"ttl,omitempty"`
	// Timestamp of when this resource is considered expired.
	ExpireTime time.Time `json:"expireTime,omitempty"`
	// The user-generated meaningful display name of the cached content.
	DisplayName string `json:"displayName,omitempty"`
	// The content to cache.
	Contents []*Content `json:"contents,omitempty"`
	// Developer set system instruction.
	SystemInstruction *Content `json:"systemInstruction,omitempty"`
	// A list of `Tools` the model may use to generate the next response.
	Tools []*Tool `json:"tools,omitempty"`
	// Configuration for the tools to use. This config is shared for all tools.
	ToolConfig *ToolConfig `json:"toolConfig,omitempty"`
}

Optional configuration for cached content creation.

func (*CreateCachedContentConfig) MarshalJSON added in v0.6.0

func (c *CreateCachedContentConfig) MarshalJSON() ([]byte, error)

func (*CreateCachedContentConfig) UnmarshalJSON added in v0.7.0

func (c *CreateCachedContentConfig) UnmarshalJSON(data []byte) error

type CreateFileConfig added in v1.0.0

type CreateFileConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

Used to override the default configuration.

type CreateFileResponse added in v1.0.0

type CreateFileResponse struct {
	// Used to retain the HTTP headers in the request
	HTTPHeaders http.Header `json:"httpHeaders,omitempty"`
}

Response for the create file method.

type DeleteCachedContentConfig added in v0.1.0

type DeleteCachedContentConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

Optional parameters for caches.delete method.

type DeleteCachedContentResponse added in v0.1.0

type DeleteCachedContentResponse struct {
}

Empty response for caches.delete method.

type DeleteFileConfig added in v1.0.0

type DeleteFileConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

Used to override the default configuration.

type DeleteFileResponse added in v1.0.0

type DeleteFileResponse struct {
}

Response for the delete file method.

type DeleteModelConfig added in v0.4.0

type DeleteModelConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

type DeleteModelResponse added in v0.4.0

type DeleteModelResponse struct {
}

type DownloadFileConfig added in v0.5.0

type DownloadFileConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

Used to override the default configuration.

type DownloadURI added in v1.0.0

type DownloadURI interface {
	// contains filtered or unexported methods
}

DownloadURI represents a resource that can be downloaded.

It is used to abstract the different types of resources that can be downloaded, such as files or videos

You can create instances that implement this interface using the following constructor functions:

  • NewDownloadURIFromFile
  • NewDownloadURIFromVideo
  • NewDownloadURIFromGeneratedVideo
  • ...

func NewDownloadURIFromFile added in v1.0.0

func NewDownloadURIFromFile(f *File) DownloadURI

NewDownloadURIFromFile creates a DownloadURI from a File.

func NewDownloadURIFromGeneratedVideo added in v1.0.0

func NewDownloadURIFromGeneratedVideo(v *GeneratedVideo) DownloadURI

NewDownloadURIFromVideo creates a DownloadURI from a GeneratedVideo.

func NewDownloadURIFromVideo added in v1.0.0

func NewDownloadURIFromVideo(v *Video) DownloadURI

NewDownloadURIFromVideo creates a DownloadURI from a Video.

type DynamicRetrievalConfig

type DynamicRetrievalConfig struct {
	// The mode of the predictor to be used in dynamic retrieval.
	Mode DynamicRetrievalConfigMode `json:"mode,omitempty"`
	// Optional. The threshold to be used in dynamic retrieval. If empty, a system default
	// value is used.
	DynamicThreshold *float32 `json:"dynamicThreshold,omitempty"`
}

Describes the options to customize dynamic retrieval.

type DynamicRetrievalConfigMode

type DynamicRetrievalConfigMode string

Config for the dynamic retrieval config mode.

const (
	// Always trigger retrieval.
	DynamicRetrievalConfigModeUnspecified DynamicRetrievalConfigMode = "MODE_UNSPECIFIED"
	// Run retrieval only when system decides it is necessary.
	DynamicRetrievalConfigModeDynamic DynamicRetrievalConfigMode = "MODE_DYNAMIC"
)

type EditImageConfig added in v0.5.0

type EditImageConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// Cloud Storage URI used to store the generated images.
	OutputGCSURI string `json:"outputGcsUri,omitempty"`
	// Description of what to discourage in the generated images.
	NegativePrompt string `json:"negativePrompt,omitempty"`
	// Number of images to generate.
	// If empty, the system will choose a default value (currently 4).
	NumberOfImages int32 `json:"numberOfImages,omitempty"`
	// Aspect ratio of the generated images.
	AspectRatio string `json:"aspectRatio,omitempty"`
	// Controls how much the model adheres to the text prompt. Large
	// values increase output and prompt alignment, but may compromise image
	// quality.
	GuidanceScale *float32 `json:"guidanceScale,omitempty"`
	// Random seed for image generation. This is not available when
	// “add_watermark“ is set to true.
	Seed *int32 `json:"seed,omitempty"`
	// Filter level for safety filtering.
	SafetyFilterLevel SafetyFilterLevel `json:"safetyFilterLevel,omitempty"`
	// Allows generation of people by the model.
	PersonGeneration PersonGeneration `json:"personGeneration,omitempty"`
	// Whether to report the safety scores of each generated image and
	// the positive prompt in the response.
	IncludeSafetyAttributes bool `json:"includeSafetyAttributes,omitempty"`
	// Whether to include the Responsible AI filter reason if the image
	// is filtered out of the response.
	IncludeRAIReason bool `json:"includeRaiReason,omitempty"`
	// Language of the text in the prompt.
	Language ImagePromptLanguage `json:"language,omitempty"`
	// MIME type of the generated image.
	OutputMIMEType string `json:"outputMimeType,omitempty"`
	// Compression quality of the generated image (for “image/jpeg“
	// only).
	OutputCompressionQuality *int32 `json:"outputCompressionQuality,omitempty"`
	// Describes the editing mode for the request.
	EditMode EditMode `json:"editMode,omitempty"`
	// The number of sampling steps. A higher value has better image
	// quality, while a lower value has better latency.
	BaseSteps *int32 `json:"baseSteps,omitempty"`
}

Configuration for editing an image.

type EditImageResponse added in v0.5.0

type EditImageResponse struct {
	// Generated images.
	GeneratedImages []*GeneratedImage `json:"generatedImages,omitempty"`
}

Response for the request to edit an image.

type EditMode added in v0.5.0

type EditMode string

Enum representing the Imagen 3 Edit mode.

const (
	EditModeDefault           EditMode = "EDIT_MODE_DEFAULT"
	EditModeInpaintRemoval    EditMode = "EDIT_MODE_INPAINT_REMOVAL"
	EditModeInpaintInsertion  EditMode = "EDIT_MODE_INPAINT_INSERTION"
	EditModeOutpaint          EditMode = "EDIT_MODE_OUTPAINT"
	EditModeControlledEditing EditMode = "EDIT_MODE_CONTROLLED_EDITING"
	EditModeStyle             EditMode = "EDIT_MODE_STYLE"
	EditModeBgswap            EditMode = "EDIT_MODE_BGSWAP"
	EditModeProductImage      EditMode = "EDIT_MODE_PRODUCT_IMAGE"
)

type EmbedContentConfig added in v0.5.0

type EmbedContentConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// Type of task for which the embedding will be used.
	TaskType string `json:"taskType,omitempty"`
	// Title for the text. Only applicable when TaskType is
	// `RETRIEVAL_DOCUMENT`.
	Title string `json:"title,omitempty"`
	// Reduced dimension for the output embedding. If set,
	// excessive values in the output embedding are truncated from the end.
	// Supported by newer models since 2024 only. You cannot set this value if
	// using the earlier model (`models/embedding-001`).
	OutputDimensionality *int32 `json:"outputDimensionality,omitempty"`
	// Vertex API only. The MIME type of the input.
	MIMEType string `json:"mimeType,omitempty"`
	// Vertex API only. Whether to silently truncate inputs longer than
	// the max sequence length. If this option is set to false, oversized inputs
	// will lead to an INVALID_ARGUMENT error, similar to other text APIs.
	AutoTruncate bool `json:"autoTruncate,omitempty"`
}

Optional parameters for the EmbedContent method.

type EmbedContentMetadata added in v0.5.0

type EmbedContentMetadata struct {
	// Vertex API only. The total number of billable characters included
	// in the request.
	BillableCharacterCount int32 `json:"billableCharacterCount,omitempty"`
}

Request-level metadata for the Vertex Embed Content API.

type EmbedContentResponse added in v0.5.0

type EmbedContentResponse struct {
	// The embeddings for each request, in the same order as provided in
	// the batch request.
	Embeddings []*ContentEmbedding `json:"embeddings,omitempty"`
	// Vertex API only. Metadata about the request.
	Metadata *EmbedContentMetadata `json:"metadata,omitempty"`
}

Response for the embed_content method.

type EndSensitivity added in v1.0.0

type EndSensitivity string

End of speech sensitivity.

const (
	// The default is END_SENSITIVITY_LOW.
	EndSensitivityUnspecified EndSensitivity = "END_SENSITIVITY_UNSPECIFIED"
	// Automatic detection ends speech more often.
	EndSensitivityHigh EndSensitivity = "END_SENSITIVITY_HIGH"
	// Automatic detection ends speech less often.
	EndSensitivityLow EndSensitivity = "END_SENSITIVITY_LOW"
)

type Endpoint added in v0.4.0

type Endpoint struct {
	// Resource name of the endpoint.
	Name string `json:"name,omitempty"`
	// ID of the model that's deployed to the endpoint.
	DeployedModelID string `json:"deployedModelId,omitempty"`
}

An endpoint where models are deployed.

type ExecutableCode

type ExecutableCode struct {
	// Required. The code to be executed.
	Code string `json:"code,omitempty"`
	// Required. Programming language of the `code`.
	Language Language `json:"language,omitempty"`
}

Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the FunctionDeclaration tool and FunctionCallingConfig mode is set to [Mode.CODE].

type FeatureSelectionPreference added in v1.1.0

type FeatureSelectionPreference string

Options for feature selection preference.

const (
	FeatureSelectionPreferenceUnspecified       FeatureSelectionPreference = "FEATURE_SELECTION_PREFERENCE_UNSPECIFIED"
	FeatureSelectionPreferencePrioritizeQuality FeatureSelectionPreference = "PRIORITIZE_QUALITY"
	FeatureSelectionPreferenceBalanced          FeatureSelectionPreference = "BALANCED"
	FeatureSelectionPreferencePrioritizeCost    FeatureSelectionPreference = "PRIORITIZE_COST"
)

type FetchPredictOperationConfig added in v0.7.0

type FetchPredictOperationConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

type File added in v1.0.0

type File struct {
	// The `File` resource name. The ID (name excluding the "files/" prefix) can contain
	// up to 40 characters that are lowercase alphanumeric or dashes (-). The ID cannot
	// start or end with a dash. If the name is empty on create, a unique name will be generated.
	// Example: `files/123-456`
	Name string `json:"name,omitempty"`
	// Optional. The human-readable display name for the `File`. The display name must be
	// no more than 512 characters in length, including spaces. Example: 'Welcome Image'
	DisplayName string `json:"displayName,omitempty"`
	// Output only. MIME type of the file.
	MIMEType string `json:"mimeType,omitempty"`
	// Output only. Size of the file in bytes.
	SizeBytes *int64 `json:"sizeBytes,omitempty"`
	// Output only. The timestamp of when the `File` was created.
	CreateTime time.Time `json:"createTime,omitempty"`
	// Output only. The timestamp of when the `File` will be deleted. Only set if the `File`
	// is scheduled to expire.
	ExpirationTime time.Time `json:"expirationTime,omitempty"`
	// Output only. The timestamp of when the `File` was last updated.
	UpdateTime time.Time `json:"updateTime,omitempty"`
	// Output only. SHA-256 hash of the uploaded bytes. The hash value is encoded in base64
	// format.
	Sha256Hash string `json:"sha256Hash,omitempty"`
	// Output only. The URI of the `File`.
	URI string `json:"uri,omitempty"`
	// Output only. The URI of the `File`, only set for downloadable (generated) files.
	DownloadURI string `json:"downloadUri,omitempty"`
	// Output only. Processing state of the File.
	State FileState `json:"state,omitempty"`
	// Output only. The source of the `File`.
	Source FileSource `json:"source,omitempty"`
	// Output only. Metadata for a video.
	VideoMetadata map[string]any `json:"videoMetadata,omitempty"`
	// Output only. Error status if File processing failed.
	Error *FileStatus `json:"error,omitempty"`
}

A file uploaded to the API.

func (*File) MarshalJSON added in v1.0.0

func (f *File) MarshalJSON() ([]byte, error)

func (*File) UnmarshalJSON added in v1.0.0

func (f *File) UnmarshalJSON(data []byte) error

type FileData

type FileData struct {
	// Required. URI.
	FileURI string `json:"fileUri,omitempty"`
	// Required. The IANA standard MIME type of the source data.
	MIMEType string `json:"mimeType,omitempty"`
}

URI based data.

type FileSource added in v1.0.0

type FileSource string

Source of the File.

const (
	FileSourceUnspecified FileSource = "SOURCE_UNSPECIFIED"
	FileSourceUploaded    FileSource = "UPLOADED"
	FileSourceGenerated   FileSource = "GENERATED"
)

type FileState added in v1.0.0

type FileState string

State for the lifecycle of a File.

const (
	FileStateUnspecified FileState = "STATE_UNSPECIFIED"
	FileStateProcessing  FileState = "PROCESSING"
	FileStateActive      FileState = "ACTIVE"
	FileStateFailed      FileState = "FAILED"
)

type FileStatus added in v1.0.0

type FileStatus struct {
	// A list of messages that carry the error details. There is a common set of message
	// types for APIs to use.
	Details []map[string]any `json:"details,omitempty"`
	// A list of messages that carry the error details. There is a common set of message
	// types for APIs to use.
	Message string `json:"message,omitempty"`
	// The status code. 0 for OK, 1 for CANCELLED
	Code *int32 `json:"code,omitempty"`
}

Status of a File that uses a common error model.

type Files added in v1.0.0

type Files struct {
	// contains filtered or unexported fields
}

func (Files) All added in v1.0.0

func (m Files) All(ctx context.Context) iter.Seq2[*File, error]

All retrieves all files resources.

This method handles pagination internally, making multiple API calls as needed to fetch all entries. It returns an iterator that yields each file entry one by one. You do not need to manage pagination tokens or make multiple calls to retrieve all data.

func (Files) Delete added in v1.0.0

func (m Files) Delete(ctx context.Context, name string, config *DeleteFileConfig) (*DeleteFileResponse, error)

func (Files) Download added in v1.0.0

func (m Files) Download(ctx context.Context, uri DownloadURI, config *DownloadFileConfig) ([]byte, error)

Download function downloads a file from the specified URI. If the URI refers to a video(Video, GeneratedVideo), the video bytes will be populated to the video's VideoBytes field.

func (Files) Get added in v1.0.0

func (m Files) Get(ctx context.Context, name string, config *GetFileConfig) (*File, error)

func (Files) List added in v1.0.0

func (m Files) List(ctx context.Context, config *ListFilesConfig) (Page[File], error)

List retrieves a paginated list of files resources.

func (Files) Upload added in v1.0.0

func (m Files) Upload(ctx context.Context, r io.Reader, config *UploadFileConfig) (*File, error)

Upload copies the contents of the given io.Reader to file storage associated with the service, and returns information about the resulting file.

func (Files) UploadFromPath added in v1.0.0

func (m Files) UploadFromPath(ctx context.Context, path string, config *UploadFileConfig) (*File, error)

UploadFromPath uploads a file from the specified path and returns information about the resulting file.

type FinishReason

type FinishReason string

The reason why the model stopped generating tokens. If empty, the model has not stopped generating the tokens.

const (
	// The finish reason is unspecified.
	FinishReasonUnspecified FinishReason = "FINISH_REASON_UNSPECIFIED"
	// Token generation reached a natural stopping point or a configured stop sequence.
	FinishReasonStop FinishReason = "STOP"
	// Token generation reached the configured maximum output tokens.
	FinishReasonMaxTokens FinishReason = "MAX_TOKENS"
	// Token generation stopped because the content potentially contains safety violations.
	// NOTE: When streaming, [content][] is empty if content filters blocks the output.
	FinishReasonSafety FinishReason = "SAFETY"
	// The token generation stopped because of potential recitation.
	FinishReasonRecitation FinishReason = "RECITATION"
	// All other reasons that stopped the token generation.
	FinishReasonOther FinishReason = "OTHER"
	// Token generation stopped because the content contains forbidden terms.
	FinishReasonBlocklist FinishReason = "BLOCKLIST"
	// Token generation stopped for potentially containing prohibited content.
	FinishReasonProhibitedContent FinishReason = "PROHIBITED_CONTENT"
	// Token generation stopped because the content potentially contains Sensitive Personally
	// Identifiable Information (SPII).
	FinishReasonSPII FinishReason = "SPII"
	// The function call generated by the model is invalid.
	FinishReasonMalformedFunctionCall FinishReason = "MALFORMED_FUNCTION_CALL"
	// Token generation stopped because generated images have safety violations.
	FinishReasonImageSafety FinishReason = "IMAGE_SAFETY"
)

type FunctionCall

type FunctionCall struct {
	// The unique ID of the function call. If populated, the client to execute the
	// `function_call` and return the response with the matching `id`.
	ID string `json:"id,omitempty"`
	// Optional. Required. The function parameters and values in JSON object format. See
	// [FunctionDeclaration.parameters] for parameter details.
	Args map[string]any `json:"args,omitempty"`
	// Required. The name of the function to call. Matches [FunctionDeclaration.Name].
	Name string `json:"name,omitempty"`
}

A function call.

type FunctionCallingConfig

type FunctionCallingConfig struct {
	// Optional. Function calling mode.
	Mode FunctionCallingConfigMode `json:"mode,omitempty"`
	// Optional. Function names to call. Only set when the Mode is ANY. Function names should
	// match [FunctionDeclaration.Name]. With mode set to ANY, model will predict a function
	// call from the set of function names provided.
	AllowedFunctionNames []string `json:"allowedFunctionNames,omitempty"`
}

Function calling config.

type FunctionCallingConfigMode

type FunctionCallingConfigMode string

Config for the function calling config mode.

const (
	// The function calling config mode is unspecified. Should not be used.
	FunctionCallingConfigModeUnspecified FunctionCallingConfigMode = "MODE_UNSPECIFIED"
	// Default model behavior, model decides to predict either function calls or natural
	// language response.
	FunctionCallingConfigModeAuto FunctionCallingConfigMode = "AUTO"
	// Model is constrained to always predicting function calls only. If "allowed_function_names"
	// are set, the predicted function calls will be limited to any one of "allowed_function_names",
	// else the predicted function calls will be any one of the provided "function_declarations".
	FunctionCallingConfigModeAny FunctionCallingConfigMode = "ANY"
	// Model will not predict any function calls. Model behavior is same as when not passing
	// any function declarations.
	FunctionCallingConfigModeNone FunctionCallingConfigMode = "NONE"
)

type FunctionDeclaration

type FunctionDeclaration struct {
	// Describes the output from the function in the OpenAPI JSON Schema
	// Object format.
	Response *Schema `json:"response,omitempty"`
	// Optional. Description and purpose of the function. Model uses it to decide how and
	// whether to call the function.
	Description string `json:"description,omitempty"`
	// Required. The name of the function to call. Must start with a letter or an underscore.
	// Must be a-z, A-Z, 0-9, or contain underscores, dots and dashes, with a maximum length
	// of 64.
	Name string `json:"name,omitempty"`
	// Optional. Describes the parameters to this function in JSON Schema Object format.
	// Reflects the Open API 3.03 Parameter Object. string Key: the name of the parameter.
	// Parameter names are case sensitive. Schema Value: the Schema defining the type used
	// for the parameter. For function with no parameters, this can be left unset. Parameter
	// names must start with a letter or an underscore and must only contain chars a-z,
	// A-Z, 0-9, or underscores with a maximum length of 64. Example with 1 required and
	// 1 optional parameter: type: OBJECT properties: param1: type: STRING param2: type:
	// INTEGER required: - param1
	Parameters *Schema `json:"parameters,omitempty"`
}

Defines a function that the model can generate JSON inputs for. The inputs are based on `OpenAPI 3.0 specifications <https://spec.openapis.org/oas/v3.0.3>`_.

type FunctionResponse

type FunctionResponse struct {
	// The ID of the function call this response is for. Populated by the client
	// to match the corresponding function call `id`.
	ID string `json:"id,omitempty"`
	// Required. The name of the function to call. Matches [FunctionDeclaration.name] and
	// [FunctionCall.name].
	Name string `json:"name,omitempty"`
	// Required. The function response in JSON object format. Use "output" key to specify
	// function output and "error" key to specify error details (if any). If "output" and
	// "error" keys are not specified, then whole "response" is treated as function output.
	Response map[string]any `json:"response,omitempty"`
}

A function response.

type GenerateContentConfig

type GenerateContentConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// Instructions for the model to steer it toward better performance.
	// For example, "Answer as concisely as possible" or "Don't use technical
	// terms in your response".
	SystemInstruction *Content `json:"systemInstruction,omitempty"`
	// Value that controls the degree of randomness in token selection.
	// Lower temperatures are good for prompts that require a less open-ended or
	// creative response, while higher temperatures can lead to more diverse or
	// creative results.
	Temperature *float32 `json:"temperature,omitempty"`
	// Tokens are selected from the most to least probable until the sum
	// of their probabilities equals this value. Use a lower value for less
	// random responses and a higher value for more random responses.
	TopP *float32 `json:"topP,omitempty"`
	// For each token selection step, the “top_k“ tokens with the
	// highest probabilities are sampled. Then tokens are further filtered based
	// on “top_p“ with the final token selected using temperature sampling. Use
	// a lower number for less random responses and a higher number for more
	// random responses.
	TopK *float32 `json:"topK,omitempty"`
	// Number of response variations to return.
	// If empty, the system will choose a default value (currently 1).
	CandidateCount int32 `json:"candidateCount,omitempty"`
	// Maximum number of tokens that can be generated in the response.
	// If empty, API will use a default value. The default value varies by model.
	MaxOutputTokens int32 `json:"maxOutputTokens,omitempty"`
	// List of strings that tells the model to stop generating text if one
	// of the strings is encountered in the response.
	StopSequences []string `json:"stopSequences,omitempty"`
	// Whether to return the log probabilities of the tokens that were
	// chosen by the model at each step.
	ResponseLogprobs bool `json:"responseLogprobs,omitempty"`
	// Number of top candidate tokens to return the log probabilities for
	// at each generation step.
	Logprobs *int32 `json:"logprobs,omitempty"`
	// Positive values penalize tokens that already appear in the
	// generated text, increasing the probability of generating more diverse
	// content.
	PresencePenalty *float32 `json:"presencePenalty,omitempty"`
	// Positive values penalize tokens that repeatedly appear in the
	// generated text, increasing the probability of generating more diverse
	// content.
	FrequencyPenalty *float32 `json:"frequencyPenalty,omitempty"`
	// When “seed“ is fixed to a specific number, the model makes a best
	// effort to provide the same response for repeated requests. By default, a
	// random number is used.
	Seed *int32 `json:"seed,omitempty"`
	// Output response media type of the generated candidate text.
	ResponseMIMEType string `json:"responseMimeType,omitempty"`
	// Schema that the generated candidate text must adhere to.
	ResponseSchema *Schema `json:"responseSchema,omitempty"`
	// Configuration for model router requests.
	RoutingConfig *GenerationConfigRoutingConfig `json:"routingConfig,omitempty"`
	// Configuration for model selection.
	ModelSelectionConfig *ModelSelectionConfig `json:"modelSelectionConfig,omitempty"`
	// Safety settings in the request to block unsafe content in the
	// response.
	SafetySettings []*SafetySetting `json:"safetySettings,omitempty"`
	// Code that enables the system to interact with external systems to
	// perform an action outside of the knowledge and scope of the model.
	Tools []*Tool `json:"tools,omitempty"`
	// Associates model output to a specific function call.
	ToolConfig *ToolConfig `json:"toolConfig,omitempty"`
	// Labels with user-defined metadata to break down billed charges.
	Labels map[string]string `json:"labels,omitempty"`
	// Resource name of a context cache that can be used in subsequent
	// requests.
	CachedContent string `json:"cachedContent,omitempty"`
	// The requested modalities of the response. Represents the set of
	// modalities that the model can return.
	ResponseModalities []string `json:"responseModalities,omitempty"`
	// If specified, the media resolution specified will be used.
	MediaResolution MediaResolution `json:"mediaResolution,omitempty"`
	// The speech generation configuration.
	SpeechConfig *SpeechConfig `json:"speechConfig,omitempty"`
	// If enabled, audio timestamp will be included in the request to the
	// model.
	AudioTimestamp bool `json:"audioTimestamp,omitempty"`
	// The thinking features configuration.
	ThinkingConfig *ThinkingConfig `json:"thinkingConfig,omitempty"`
}

Optional model configuration parameters. For more information, see `Content generation parameters <https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/content-generation-parameters>`_.

type GenerateContentResponse

type GenerateContentResponse struct {
	// Response variations returned by the model.
	Candidates []*Candidate `json:"candidates,omitempty"`
	// Timestamp when the request is made to the server.
	CreateTime time.Time `json:"createTime,omitempty"`
	// Identifier for each response.
	ResponseID string `json:"responseId,omitempty"`
	// Output only. The model version used to generate the response.
	ModelVersion string `json:"modelVersion,omitempty"`
	// Output only. Content filter results for a prompt sent in the request. Note: Sent
	// only in the first stream chunk. Only happens when no candidates were generated due
	// to content violations.
	PromptFeedback *GenerateContentResponsePromptFeedback `json:"promptFeedback,omitempty"`
	// Usage metadata about the response(s).
	UsageMetadata *GenerateContentResponseUsageMetadata `json:"usageMetadata,omitempty"`
}

Response message for PredictionService.GenerateContent.

func (*GenerateContentResponse) CodeExecutionResult added in v0.5.0

func (r *GenerateContentResponse) CodeExecutionResult() string

CodeExecutionResult returns the code execution result in the GenerateContentResponse.

func (*GenerateContentResponse) ExecutableCode added in v0.5.0

func (r *GenerateContentResponse) ExecutableCode() string

ExecutableCode returns the executable code in the GenerateContentResponse.

func (*GenerateContentResponse) FunctionCalls added in v0.1.0

func (r *GenerateContentResponse) FunctionCalls() []*FunctionCall

FunctionCalls returns the list of function calls in the GenerateContentResponse.

func (*GenerateContentResponse) MarshalJSON added in v0.6.0

func (c *GenerateContentResponse) MarshalJSON() ([]byte, error)

func (*GenerateContentResponse) Text added in v0.1.0

func (r *GenerateContentResponse) Text() string

Text concatenates all the text parts in the GenerateContentResponse.

type GenerateContentResponsePromptFeedback

type GenerateContentResponsePromptFeedback struct {
	// Output only. Blocked reason.
	BlockReason BlockedReason `json:"blockReason,omitempty"`
	// Output only. A readable block reason message.
	BlockReasonMessage string `json:"blockReasonMessage,omitempty"`
	// Output only. Safety ratings.
	SafetyRatings []*SafetyRating `json:"safetyRatings,omitempty"`
}

Content filter results for a prompt sent in the request.

type GenerateContentResponseUsageMetadata

type GenerateContentResponseUsageMetadata struct {
	// Output only. List of modalities of the cached content in the request input.
	CacheTokensDetails []*ModalityTokenCount `json:"cacheTokensDetails,omitempty"`
	// Output only. Number of tokens in the cached part in the input (the cached content).
	CachedContentTokenCount int32 `json:"cachedContentTokenCount,omitempty"`
	// Number of tokens in the response(s). This includes all the generated response candidates.
	CandidatesTokenCount int32 `json:"candidatesTokenCount,omitempty"`
	// Output only. List of modalities that were returned in the response.
	CandidatesTokensDetails []*ModalityTokenCount `json:"candidatesTokensDetails,omitempty"`
	// Number of tokens in the prompt. When cached_content is set, this is still the total
	// effective prompt size meaning this includes the number of tokens in the cached content.
	PromptTokenCount int32 `json:"promptTokenCount,omitempty"`
	// Output only. List of modalities that were processed in the request input.
	PromptTokensDetails []*ModalityTokenCount `json:"promptTokensDetails,omitempty"`
	// Output only. Number of tokens present in thoughts output.
	ThoughtsTokenCount int32 `json:"thoughtsTokenCount,omitempty"`
	// Output only. Number of tokens present in tool-use prompt(s).
	ToolUsePromptTokenCount int32 `json:"toolUsePromptTokenCount,omitempty"`
	// Output only. List of modalities that were processed for tool-use request inputs.
	ToolUsePromptTokensDetails []*ModalityTokenCount `json:"toolUsePromptTokensDetails,omitempty"`
	// Total token count for prompt, response candidates, and tool-use prompts (if present).
	TotalTokenCount int32 `json:"totalTokenCount,omitempty"`
	// Output only. Traffic type. This shows whether a request consumes Pay-As-You-Go or
	// Provisioned Throughput quota.
	TrafficType TrafficType `json:"trafficType,omitempty"`
}

Usage metadata about response(s).

type GenerateImagesConfig added in v0.1.0

type GenerateImagesConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// Cloud Storage URI used to store the generated images.
	OutputGCSURI string `json:"outputGcsUri,omitempty"`
	// Description of what to discourage in the generated images.
	NegativePrompt string `json:"negativePrompt,omitempty"`
	// Number of images to generate.
	// If empty, the system will choose a default value (currently 4).
	NumberOfImages int32 `json:"numberOfImages,omitempty"`
	// Aspect ratio of the generated images.
	AspectRatio string `json:"aspectRatio,omitempty"`
	// Controls how much the model adheres to the text prompt. Large
	// values increase output and prompt alignment, but may compromise image
	// quality.
	GuidanceScale *float32 `json:"guidanceScale,omitempty"`
	// Random seed for image generation. This is not available when
	// “add_watermark“ is set to true.
	Seed *int32 `json:"seed,omitempty"`
	// Filter level for safety filtering.
	SafetyFilterLevel SafetyFilterLevel `json:"safetyFilterLevel,omitempty"`
	// Allows generation of people by the model.
	PersonGeneration PersonGeneration `json:"personGeneration,omitempty"`
	// Whether to report the safety scores of each generated image and
	// the positive prompt in the response.
	IncludeSafetyAttributes bool `json:"includeSafetyAttributes,omitempty"`
	// Whether to include the Responsible AI filter reason if the image
	// is filtered out of the response.
	IncludeRAIReason bool `json:"includeRaiReason,omitempty"`
	// Language of the text in the prompt.
	Language ImagePromptLanguage `json:"language,omitempty"`
	// MIME type of the generated image.
	OutputMIMEType string `json:"outputMimeType,omitempty"`
	// Compression quality of the generated image (for “image/jpeg“
	// only).
	OutputCompressionQuality *int32 `json:"outputCompressionQuality,omitempty"`
	// Whether to add a watermark to the generated images.
	AddWatermark bool `json:"addWatermark,omitempty"`
	// Whether to use the prompt rewriting logic.
	EnhancePrompt bool `json:"enhancePrompt,omitempty"`
}

The configuration for generating images. You can find API default values and more details at VertexAI: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/imagen-api. GeminiAPI: https://ai.google.dev/gemini-api/docs/imagen#imagen-model

type GenerateImagesResponse added in v0.1.0

type GenerateImagesResponse struct {
	// List of generated images.
	GeneratedImages []*GeneratedImage `json:"generatedImages,omitempty"`
	// Safety attributes of the positive prompt. Only populated if
	// “include_safety_attributes“ is set to True.
	PositivePromptSafetyAttributes *SafetyAttributes `json:"positivePromptSafetyAttributes,omitempty"`
}

The output images response.

type GenerateVideosConfig added in v0.7.0

type GenerateVideosConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// Number of output videos. If empty, the system will choose a default value.
	NumberOfVideos int32 `json:"numberOfVideos,omitempty"`
	// The GCS bucket where to save the generated videos.
	OutputGCSURI string `json:"outputGcsUri,omitempty"`
	// Frames per second for video generation.
	FPS *int32 `json:"fps,omitempty"`
	// Duration of the clip for video generation in seconds.
	DurationSeconds *int32 `json:"durationSeconds,omitempty"`
	// The RNG seed. If RNG seed is exactly same for each request with unchanged inputs,
	// the prediction results will be consistent. Otherwise, a random RNG seed will be used
	// each time to produce a different result.
	Seed *int32 `json:"seed,omitempty"`
	// The aspect ratio for the generated video. 16:9 (landscape) and 9:16 (portrait) are
	// supported.
	AspectRatio string `json:"aspectRatio,omitempty"`
	// The resolution for the generated video. 1280x720, 1920x1080 are supported.
	Resolution string `json:"resolution,omitempty"`
	// Whether allow to generate person videos, and restrict to specific ages. Supported
	// values are: dont_allow, allow_adult.
	PersonGeneration string `json:"personGeneration,omitempty"`
	// The pubsub topic where to publish the video generation progress.
	PubsubTopic string `json:"pubsubTopic,omitempty"`
	// Optional field in addition to the text content. Negative prompts can be explicitly
	// stated here to help generate the video.
	NegativePrompt string `json:"negativePrompt,omitempty"`
	// Whether to use the prompt rewriting logic.
	EnhancePrompt bool `json:"enhancePrompt,omitempty"`
}

You can find API default values and more details at VertexAI: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/veo-video-generation.

type GenerateVideosOperation added in v0.7.0

type GenerateVideosOperation struct {
	// The server-assigned name, which is only unique within the same service that originally
	// returns it. If you use the default HTTP mapping, the `name` should be a resource
	// name ending with `operations/{unique_id}`.
	Name string `json:"name,omitempty"`
	// Service-specific metadata associated with the operation. It typically contains progress
	// information and common metadata such as create time. Some services might not provide
	// such metadata. Any method that returns a long-running operation should document the
	// metadata type, if any.
	Metadata map[string]any `json:"metadata,omitempty"`
	// If the value is `false`, it means the operation is still in progress. If `true`,
	// the operation is completed, and either `error` or `response` is available.
	Done bool `json:"done,omitempty"`
	// The error result of the operation in case of failure or cancellation.
	Error map[string]any `json:"error,omitempty"`
	// The generated videos.
	Response *GenerateVideosResponse `json:"response,omitempty"`
}

A video generation operation.

type GenerateVideosResponse added in v0.7.0

type GenerateVideosResponse struct {
	// List of the generated videos
	GeneratedVideos []*GeneratedVideo `json:"generatedVideos,omitempty"`
	// Returns if any videos were filtered due to RAI policies.
	RAIMediaFilteredCount int32 `json:"raiMediaFilteredCount,omitempty"`
	// Returns RAI failure reasons if any.
	RAIMediaFilteredReasons []string `json:"raiMediaFilteredReasons,omitempty"`
}

Response with generated videos.

type GeneratedImage added in v0.1.0

type GeneratedImage struct {
	// The output image data.
	Image *Image `json:"image,omitempty"`
	// Responsible AI filter reason if the image is filtered out of the
	// response.
	RAIFilteredReason string `json:"raiFilteredReason,omitempty"`
	// Safety attributes of the image. Lists of RAI categories and their
	// scores of each content.
	SafetyAttributes *SafetyAttributes `json:"safetyAttributes,omitempty"`
	// The rewritten prompt used for the image generation if the prompt
	// enhancer is enabled.
	EnhancedPrompt string `json:"enhancedPrompt,omitempty"`
}

An output image.

type GeneratedVideo added in v0.7.0

type GeneratedVideo struct {
	// The output video
	Video *Video `json:"video,omitempty"`
}

A generated video.

type GenerationConfig

type GenerationConfig struct {
	// Optional. If enabled, audio timestamp will be included in the request to the model.
	AudioTimestamp bool `json:"audioTimestamp,omitempty"`
	// Optional. Number of candidates to generate. If empty, the system will choose a default
	// value (currently 1).
	CandidateCount int32 `json:"candidateCount,omitempty"`
	// Optional. Frequency penalties.
	FrequencyPenalty *float32 `json:"frequencyPenalty,omitempty"`
	// Optional. Logit probabilities.
	Logprobs *int32 `json:"logprobs,omitempty"`
	// Optional. The maximum number of output tokens to generate per message. If empty,
	// API will use a default value. The default value varies by model.
	MaxOutputTokens int32 `json:"maxOutputTokens,omitempty"`
	// Optional. If specified, the media resolution specified will be used.
	MediaResolution MediaResolution `json:"mediaResolution,omitempty"`
	// Optional. Positive penalties.
	PresencePenalty *float32 `json:"presencePenalty,omitempty"`
	// Optional. If true, export the logprobs results in response.
	ResponseLogprobs bool `json:"responseLogprobs,omitempty"`
	// Optional. Output response mimetype of the generated candidate text. Supported mimetype:
	// - `text/plain`: (default) Text output. - `application/json`: JSON response in the
	// candidates. The model needs to be prompted to output the appropriate response type,
	// otherwise the behavior is undefined. This is a preview feature.
	ResponseMIMEType string `json:"responseMimeType,omitempty"`
	// Optional. The `Schema` object allows the definition of input and output data types.
	// These types can be objects, but also primitives and arrays. Represents a select subset
	// of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If
	// set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`:
	// Schema for JSON response.
	ResponseSchema *Schema `json:"responseSchema,omitempty"`
	// Optional. Routing configuration.
	RoutingConfig *GenerationConfigRoutingConfig `json:"routingConfig,omitempty"`
	// Optional. Seed.
	Seed *int32 `json:"seed,omitempty"`
	// Optional. Stop sequences.
	StopSequences []string `json:"stopSequences,omitempty"`
	// Optional. Controls the randomness of predictions.
	Temperature *float32 `json:"temperature,omitempty"`
	// Optional. If specified, top-k sampling will be used.
	TopK *float32 `json:"topK,omitempty"`
	// Optional. If specified, nucleus sampling will be used.
	TopP *float32 `json:"topP,omitempty"`
}

Generation config. You can find API default values and more details at https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#generationconfig and https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/content-generation-parameters.

type GenerationConfigRoutingConfig

type GenerationConfigRoutingConfig struct {
	// Automated routing.
	AutoMode *GenerationConfigRoutingConfigAutoRoutingMode `json:"autoMode,omitempty"`
	// Manual routing.
	ManualMode *GenerationConfigRoutingConfigManualRoutingMode `json:"manualMode,omitempty"`
}

The configuration for routing the request to a specific model.

type GenerationConfigRoutingConfigAutoRoutingMode

type GenerationConfigRoutingConfigAutoRoutingMode struct {
	// The model routing preference.
	ModelRoutingPreference string `json:"modelRoutingPreference,omitempty"`
}

When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference.

type GenerationConfigRoutingConfigManualRoutingMode

type GenerationConfigRoutingConfigManualRoutingMode struct {
	// The model name to use. Only the public LLM models are accepted. e.g. 'gemini-1.5-pro-001'.
	ModelName string `json:"modelName,omitempty"`
}

When manual routing is set, the specified model will be used directly.

type GetCachedContentConfig added in v0.1.0

type GetCachedContentConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

Optional parameters for caches.get method.

type GetFileConfig added in v1.0.0

type GetFileConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

Used to override the default configuration.

type GetModelConfig added in v0.4.0

type GetModelConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

Optional parameters for models.get method.

type GetOperationConfig added in v0.7.0

type GetOperationConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
}

type GoogleSearch

type GoogleSearch struct {
}

Tool to support Google Search in Model. Powered by Google.

type GoogleSearchRetrieval

type GoogleSearchRetrieval struct {
	// Specifies the dynamic retrieval configuration for the given source.
	DynamicRetrievalConfig *DynamicRetrievalConfig `json:"dynamicRetrievalConfig,omitempty"`
}

Tool to retrieve public web data for grounding, powered by Google.

type GroundingChunk

type GroundingChunk struct {
	// Grounding chunk from context retrieved by the retrieval tools.
	RetrievedContext *GroundingChunkRetrievedContext `json:"retrievedContext,omitempty"`
	// Grounding chunk from the web.
	Web *GroundingChunkWeb `json:"web,omitempty"`
}

Grounding chunk.

type GroundingChunkRetrievedContext

type GroundingChunkRetrievedContext struct {
	// Text of the attribution.
	Text string `json:"text,omitempty"`
	// Title of the attribution.
	Title string `json:"title,omitempty"`
	// URI reference of the attribution.
	URI string `json:"uri,omitempty"`
}

Chunk from context retrieved by the retrieval tools.

type GroundingChunkWeb

type GroundingChunkWeb struct {
	// Domain of the (original) URI.
	Domain string `json:"domain,omitempty"`
	// Title of the chunk.
	Title string `json:"title,omitempty"`
	// URI reference of the chunk.
	URI string `json:"uri,omitempty"`
}

Chunk from the web.

type GroundingMetadata

type GroundingMetadata struct {
	// List of supporting references retrieved from specified grounding source.
	GroundingChunks []*GroundingChunk `json:"groundingChunks,omitempty"`
	// Optional. List of grounding support.
	GroundingSupports []*GroundingSupport `json:"groundingSupports,omitempty"`
	// Optional. Output only. Retrieval metadata.
	RetrievalMetadata *RetrievalMetadata `json:"retrievalMetadata,omitempty"`
	// Optional. Queries executed by the retrieval tools.
	RetrievalQueries []string `json:"retrievalQueries,omitempty"`
	// Optional. Google search entry for the following-up web searches.
	SearchEntryPoint *SearchEntryPoint `json:"searchEntryPoint,omitempty"`
	// Optional. Web search queries for the following-up web search.
	WebSearchQueries []string `json:"webSearchQueries,omitempty"`
}

Metadata returned to client when grounding is enabled.

type GroundingSupport

type GroundingSupport struct {
	// Confidence score of the support references. Ranges from 0 to 1. 1 is the most confident.
	// This list must have the same size as the grounding_chunk_indices.
	ConfidenceScores []float32 `json:"confidenceScores,omitempty"`
	// A list of indices (into 'grounding_chunk') specifying the citations associated with
	// the claim. For instance [1,3,4] means that grounding_chunk[1], grounding_chunk[3],
	// grounding_chunk[4] are the retrieved content attributed to the claim.
	GroundingChunkIndices []int32 `json:"groundingChunkIndices,omitempty"`
	// Segment of the content this support belongs to.
	Segment *Segment `json:"segment,omitempty"`
}

Grounding support.

type HTTPOptions added in v0.1.0

type HTTPOptions struct {
	// BaseURL specifies the base URL for the API endpoint. If empty, defaults to "https://generativelanguage.googleapis.com/"
	// for the Gemini API backend, and location-specific Vertex AI endpoint (e.g., "https://us-central1-aiplatform.googleapis.com/
	BaseURL string `json:"baseUrl,omitempty"`
	// APIVersion specifies the version of the API to use. If empty, defaults to "v1beta"
	// for Gemini API and "v1beta1" for Vertex AI.
	APIVersion string `json:"apiVersion,omitempty"`
	// Additional HTTP headers to be sent with the request.
	Headers http.Header `json:"headers,omitempty"`
}

HTTP options to be used in each of the requests.

type HarmBlockMethod

type HarmBlockMethod string

Specify if the threshold is used for probability or severity score. If not specified, the threshold is used for probability score.

const (
	// The harm block method is unspecified.
	HarmBlockMethodUnspecified HarmBlockMethod = "HARM_BLOCK_METHOD_UNSPECIFIED"
	// The harm block method uses both probability and severity scores.
	HarmBlockMethodSeverity HarmBlockMethod = "SEVERITY"
	// The harm block method uses the probability score.
	HarmBlockMethodProbability HarmBlockMethod = "PROBABILITY"
)

type HarmBlockThreshold

type HarmBlockThreshold string

The harm block threshold.

const (
	// Unspecified harm block threshold.
	HarmBlockThresholdUnspecified HarmBlockThreshold = "HARM_BLOCK_THRESHOLD_UNSPECIFIED"
	// Block low threshold and above (i.e. block more).
	HarmBlockThresholdBlockLowAndAbove HarmBlockThreshold = "BLOCK_LOW_AND_ABOVE"
	// Block medium threshold and above.
	HarmBlockThresholdBlockMediumAndAbove HarmBlockThreshold = "BLOCK_MEDIUM_AND_ABOVE"
	// Block only high threshold (i.e. block less).
	HarmBlockThresholdBlockOnlyHigh HarmBlockThreshold = "BLOCK_ONLY_HIGH"
	// Block none.
	HarmBlockThresholdBlockNone HarmBlockThreshold = "BLOCK_NONE"
	// Turn off the safety filter.
	HarmBlockThresholdOff HarmBlockThreshold = "OFF"
)

type HarmCategory

type HarmCategory string

Harm category.

const (
	// The harm category is unspecified.
	HarmCategoryUnspecified HarmCategory = "HARM_CATEGORY_UNSPECIFIED"
	// The harm category is hate speech.
	HarmCategoryHateSpeech HarmCategory = "HARM_CATEGORY_HATE_SPEECH"
	// The harm category is dangerous content.
	HarmCategoryDangerousContent HarmCategory = "HARM_CATEGORY_DANGEROUS_CONTENT"
	// The harm category is harassment.
	HarmCategoryHarassment HarmCategory = "HARM_CATEGORY_HARASSMENT"
	// The harm category is sexually explicit content.
	HarmCategorySexuallyExplicit HarmCategory = "HARM_CATEGORY_SEXUALLY_EXPLICIT"
	// The harm category is civic integrity.
	HarmCategoryCivicIntegrity HarmCategory = "HARM_CATEGORY_CIVIC_INTEGRITY"
)

type HarmProbability

type HarmProbability string

Harm probability levels in the content.

const (
	// Harm probability unspecified.
	HarmProbabilityUnspecified HarmProbability = "HARM_PROBABILITY_UNSPECIFIED"
	// Negligible level of harm.
	HarmProbabilityNegligible HarmProbability = "NEGLIGIBLE"
	// Low level of harm.
	HarmProbabilityLow HarmProbability = "LOW"
	// Medium level of harm.
	HarmProbabilityMedium HarmProbability = "MEDIUM"
	// High level of harm.
	HarmProbabilityHigh HarmProbability = "HIGH"
)

type HarmSeverity

type HarmSeverity string

Harm severity levels in the content.

const (
	// Harm severity unspecified.
	HarmSeverityUnspecified HarmSeverity = "HARM_SEVERITY_UNSPECIFIED"
	// Negligible level of harm severity.
	HarmSeverityNegligible HarmSeverity = "HARM_SEVERITY_NEGLIGIBLE"
	// Low level of harm severity.
	HarmSeverityLow HarmSeverity = "HARM_SEVERITY_LOW"
	// Medium level of harm severity.
	HarmSeverityMedium HarmSeverity = "HARM_SEVERITY_MEDIUM"
	// High level of harm severity.
	HarmSeverityHigh HarmSeverity = "HARM_SEVERITY_HIGH"
)

type Image

type Image struct {
	// The Cloud Storage URI of the image. “Image“ can contain a value
	// for this field or the “image_bytes“ field but not both.
	GCSURI string `json:"gcsUri,omitempty"`
	// The image bytes data. “Image“ can contain a value for this field
	// or the “gcs_uri“ field but not both.
	ImageBytes []byte `json:"imageBytes,omitempty"`
	// The MIME type of the image.
	MIMEType string `json:"mimeType,omitempty"`
}

An image.

type ImagePromptLanguage added in v0.1.0

type ImagePromptLanguage string

Enum that specifies the language of the text in the prompt.

const (
	ImagePromptLanguageAuto ImagePromptLanguage = "auto"
	ImagePromptLanguageEn   ImagePromptLanguage = "en"
	ImagePromptLanguageJa   ImagePromptLanguage = "ja"
	ImagePromptLanguageKo   ImagePromptLanguage = "ko"
	ImagePromptLanguageHi   ImagePromptLanguage = "hi"
)

type Language

type Language string

Programming language of the `code`.

const (
	// Unspecified language. This value should not be used.
	LanguageUnspecified Language = "LANGUAGE_UNSPECIFIED"
	// Python >= 3.10, with numpy and simpy available.
	LanguagePython Language = "PYTHON"
)

type ListCachedContentsConfig added in v0.4.0

type ListCachedContentsConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// PageSize specifies the maximum number of cached contents to return per API call.
	// If zero, the server will use a default value.
	PageSize int32 `json:"pageSize,omitempty"`
	// PageToken represents a token used for pagination in API responses. It's an opaque
	// string that should be passed to subsequent requests to retrieve the next page of
	// results. An empty PageToken typically indicates that there are no further pages available.
	PageToken string `json:"pageToken,omitempty"`
}

Config for caches.list method.

type ListCachedContentsResponse added in v0.4.0

type ListCachedContentsResponse struct {
	NextPageToken string `json:"nextPageToken,omitempty"`
	// List of cached contents.
	CachedContents []*CachedContent `json:"cachedContents,omitempty"`
}

type ListFilesConfig added in v1.0.0

type ListFilesConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// PageSize specifies the maximum number of cached contents to return per API call.
	// If zero, the server will use a default value.
	PageSize int32 `json:"pageSize,omitempty"`
	// PageToken represents a token used for pagination in API responses. It's an opaque
	// string that should be passed to subsequent requests to retrieve the next page of
	// results. An empty PageToken typically indicates that there are no further pages available.
	PageToken string `json:"pageToken,omitempty"`
}

Used to override the default configuration.

type ListFilesResponse added in v1.0.0

type ListFilesResponse struct {
	// A token to retrieve next page of results.
	NextPageToken string `json:"nextPageToken,omitempty"`
	// The list of files.
	Files []*File `json:"files,omitempty"`
}

Response for the list files method.

type ListModelsConfig added in v0.5.0

type ListModelsConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// PageSize specifies the maximum number of cached contents to return per API call.
	// If zero, the server will use a default value.
	PageSize int32 `json:"pageSize,omitempty"`
	// PageToken represents a token used for pagination in API responses. It's an opaque
	// string that should be passed to subsequent requests to retrieve the next page of
	// results. An empty PageToken typically indicates that there are no further pages available.
	PageToken string `json:"pageToken,omitempty"`

	Filter string `json:"filter,omitempty"`
	// QueryBase is a boolean flag to control whether to query base models or tuned models.
	// If nil, then SDK will use the default value Ptr(true).
	QueryBase *bool `json:"queryBase,omitempty"`
}

type ListModelsResponse added in v0.5.0

type ListModelsResponse struct {
	NextPageToken string `json:"nextPageToken,omitempty"`

	Models []*Model `json:"models,omitempty"`
}

type Live

type Live struct {
	// contains filtered or unexported fields
}

Preview. Live can be used to create a realtime connection to the API. It is initiated when creating a client. You don't need to create a new Live object. The live module is experimental.

client, _ := genai.NewClient(ctx, &genai.ClientConfig{})
session, _ := client.Live.Connect(model, &genai.LiveConnectConfig{}).

func (*Live) Connect

func (r *Live) Connect(context context.Context, model string, config *LiveConnectConfig) (*Session, error)

Preview. Connect establishes a realtime connection to the specified model with given configuration. It returns a Session object representing the connection or an error if the connection fails. The live module is experimental.

type LiveClientContent

type LiveClientContent struct {
	// The content appended to the current conversation with the model.
	// For single-turn queries, this is a single instance. For multi-turn
	// queries, this is a repeated field that contains conversation history and
	// latest request.
	Turns []*Content `json:"turns,omitempty"`
	// If true, indicates that the server content generation should start with
	// the currently accumulated prompt. Otherwise, the server will await
	// additional messages before starting generation.
	TurnComplete bool `json:"turnComplete,omitempty"`
}

Incremental update of the current conversation delivered from the client. All the content here will unconditionally be appended to the conversation history and used as part of the prompt to the model to generate content. A message here will interrupt any current model generation.

type LiveClientContentInput added in v1.0.0

type LiveClientContentInput struct {
	// The content appended to the current conversation with the model.
	// For single-turn queries, this is a single instance. For multi-turn
	// queries, this is a repeated field that contains conversation history and
	// latest request.
	Turns []*Content `json:"turns,omitempty"`
	// TurnComplete is default to true, indicating that the server content generation should
	// start with the currently accumulated prompt. If set to false, the server will await
	// additional messages, accumulating the prompt, and start generation until received a
	// TurnComplete true message.
	TurnComplete *bool `json:"turnComplete,omitempty"`
}

Preview. LiveClientContentInput is the input for [SendClientContent].

type LiveClientMessage

type LiveClientMessage struct {
	// Message to be sent by the system when connecting to the API. SDK users should not
	// send this message.
	Setup *LiveClientSetup `json:"setup,omitempty"`
	// Incremental update of the current conversation delivered from the client.
	ClientContent *LiveClientContent `json:"clientContent,omitempty"`
	// User input that is sent in real time.
	RealtimeInput *LiveClientRealtimeInput `json:"realtimeInput,omitempty"`
	// Response to a `ToolCallMessage` received from the server.
	ToolResponse *LiveClientToolResponse `json:"toolResponse,omitempty"`
}

Messages sent by the client in the API call.

type LiveClientRealtimeInput

type LiveClientRealtimeInput struct {
	// Inlined bytes data for media input.
	MediaChunks []*Blob `json:"mediaChunks,omitempty"`
	// Marks the start of user activity.
	ActivityStart *ActivityStart `json:"activityStart,omitempty"`
	// Marks the end of user activity.
	ActivityEnd *ActivityEnd `json:"activityEnd,omitempty"`
}

User input that is sent in real time. This is different from `LiveClientContent` in a few ways:

  • Can be sent continuously without interruption to model generation.
  • If there is a need to mix data interleaved across the `LiveClientContent` and the `LiveClientRealtimeInput`, server attempts to optimize for best response, but there are no guarantees.
  • End of turn is not explicitly specified, but is rather derived from user activity (for example, end of speech).
  • Even before the end of turn, the data is processed incrementally to optimize for a fast start of the response from the model.
  • Is always assumed to be the user's input (cannot be used to populate conversation history).

type LiveClientSetup

type LiveClientSetup struct {
	// The fully qualified name of the publisher model or tuned model endpoint to
	// use.
	Model string `json:"model,omitempty"`
	// The generation configuration for the session.
	// Note: only a subset of fields are supported.
	GenerationConfig *GenerationConfig `json:"generationConfig,omitempty"`
	// The user provided system instructions for the model.
	// Note: only text should be used in parts and content in each part will be
	// in a separate paragraph.
	SystemInstruction *Content `json:"systemInstruction,omitempty"`
	// A list of `Tools` the model may use to generate the next response.
	// A `Tool` is a piece of code that enables the system to interact with
	// external systems to perform an action, or set of actions, outside of
	// knowledge and scope of the model.
	Tools []*Tool `json:"tools,omitempty"`
	// Configures the realtime input behavior in BidiGenerateContent.
	RealtimeInputConfig *RealtimeInputConfig `json:"realtimeInputConfig,omitempty"`
	// Configures session resumption mechanism.
	// If included server will send SessionResumptionUpdate messages.
	SessionResumption *SessionResumptionConfig `json:"sessionResumption,omitempty"`
	// Configures context window compression mechanism.
	// If included, server will compress context window to fit into given length.
	ContextWindowCompression *ContextWindowCompressionConfig `json:"contextWindowCompression,omitempty"`
	// The transcription of the input aligns with the input audio language.
	InputAudioTranscription *AudioTranscriptionConfig `json:"inputAudioTranscription,omitempty"`
	// The transcription of the output aligns with the language code
	// specified for the output audio.
	OutputAudioTranscription *AudioTranscriptionConfig `json:"outputAudioTranscription,omitempty"`
}

Message contains configuration that will apply for the duration of the streaming session.

type LiveClientToolResponse

type LiveClientToolResponse struct {
	// The response to the function calls.
	FunctionResponses []*FunctionResponse `json:"functionResponses,omitempty"`
}

Client generated response to a `ToolCall` received from the server. Individual `FunctionResponse` objects are matched to the respective `FunctionCall` objects by the `id` field. Note that in the unary and server-streaming GenerateContent APIs function calling happens by exchanging the `Content` parts, while in the bidi GenerateContent APIs function calling happens over this dedicated set of messages.

type LiveConnectConfig

type LiveConnectConfig struct {
	// The requested modalities of the response. Represents the set of
	// modalities that the model can return. Defaults to AUDIO if not specified.
	ResponseModalities []Modality `json:"responseModalities,omitempty"`
	// Value that controls the degree of randomness in token selection.
	// Lower temperatures are good for prompts that require a less open-ended or
	// creative response, while higher temperatures can lead to more diverse or
	// creative results.
	Temperature *float32 `json:"temperature,omitempty"`
	// Tokens are selected from the most to least probable until the sum
	// of their probabilities equals this value. Use a lower value for less
	// random responses and a higher value for more random responses.
	TopP *float32 `json:"topP,omitempty"`
	// For each token selection step, the “top_k“ tokens with the
	// highest probabilities are sampled. Then tokens are further filtered based
	// on “top_p“ with the final token selected using temperature sampling. Use
	// a lower number for less random responses and a higher number for more
	// random responses.
	TopK *float32 `json:"topK,omitempty"`
	// Maximum number of tokens that can be generated in the response.
	// If empty, API will use a default value. The default value varies by model.
	MaxOutputTokens int32 `json:"maxOutputTokens,omitempty"`
	// When “seed“ is fixed to a specific number, the model makes a best
	// effort to provide the same response for repeated requests. By default, a
	// random number is used.
	Seed *int32 `json:"seed,omitempty"`
	// The speech generation configuration.
	SpeechConfig *SpeechConfig `json:"speechConfig,omitempty"`
	// The user provided system instructions for the model.
	// Note: only text should be used in parts and content in each part will be
	// in a separate paragraph.
	SystemInstruction *Content `json:"systemInstruction,omitempty"`
	// A list of `Tools` the model may use to generate the next response.
	// A `Tool` is a piece of code that enables the system to interact with
	// external systems to perform an action, or set of actions, outside of
	// knowledge and scope of the model.
	Tools []*Tool `json:"tools,omitempty"`
	// Configures session resumption mechanism.
	// If included the server will send SessionResumptionUpdate messages.
	SessionResumption *SessionResumptionConfig `json:"sessionResumption,omitempty"`
	// The transcription of the input aligns with the input audio language.
	InputAudioTranscription *AudioTranscriptionConfig `json:"inputAudioTranscription,omitempty"`
	// The transcription of the output aligns with the language code
	// specified for the output audio.
	OutputAudioTranscription *AudioTranscriptionConfig `json:"outputAudioTranscription,omitempty"`
	// Configures the realtime input behavior in BidiGenerateContent.
	RealtimeInputConfig *RealtimeInputConfig `json:"realtimeInputConfig,omitempty"`
	// Configures context window compression mechanism.
	// If included, server will compress context window to fit into given length.
	ContextWindowCompression *ContextWindowCompressionConfig `json:"contextWindowCompression,omitempty"`
}

Session config for the API connection.

type LiveRealtimeInput added in v1.0.0

type LiveRealtimeInput struct {
	Media *Blob `json:"media,omitempty"`
}

Preview. LiveRealtimeInput is the input for [SendRealtimeInput].

type LiveServerContent

type LiveServerContent struct {
	// The content that the model has generated as part of the current conversation with
	// the user.
	ModelTurn *Content `json:"modelTurn,omitempty"`
	// If true, indicates that the model is done generating. Generation will only start
	// in response to additional client messages. Can be set alongside `content`, indicating
	// that the `content` is the last in the turn.
	TurnComplete bool `json:"turnComplete,omitempty"`
	// If true, indicates that a client message has interrupted current model generation.
	// If the client is playing out the content in realtime, this is a good signal to stop
	// and empty the current queue.
	Interrupted bool `json:"interrupted,omitempty"`
	// If true, indicates that the model is done generating. When model is
	// interrupted while generating there will be no generation_complete message
	// in interrupted turn, it will go through interrupted > turn_complete.
	// When model assumes realtime playback there will be delay between
	// generation_complete and turn_complete that is caused by model
	// waiting for playback to finish. If true, indicates that the model
	// has finished generating all content. This is a signal to the client
	// that it can stop sending messages.
	GenerationComplete bool `json:"generationComplete,omitempty"`
	// Input transcription. The transcription is independent to the model
	// turn which means it doesn’t imply any ordering between transcription and
	// model turn.
	InputTranscription *Transcription `json:"inputTranscription,omitempty"`
	// Output transcription. The transcription is independent to the model
	// turn which means it doesn’t imply any ordering between transcription and
	// model turn.
	OutputTranscription *Transcription `json:"outputTranscription,omitempty"`
}

Incremental server update generated by the model in response to client messages. Content is generated as quickly as possible, and not in real time. Clients may choose to buffer and play it out in real time.

type LiveServerGoAway added in v1.0.0

type LiveServerGoAway struct {
	// The remaining time before the connection will be terminated as ABORTED. The minimal
	// time returned here is specified differently together with the rate limits for a given
	// model.
	TimeLeft time.Duration `json:"timeLeft,omitempty"`
}

Server will not be able to service client soon.

func (*LiveServerGoAway) MarshalJSON added in v1.0.0

func (c *LiveServerGoAway) MarshalJSON() ([]byte, error)

func (*LiveServerGoAway) UnmarshalJSON added in v1.0.0

func (c *LiveServerGoAway) UnmarshalJSON(data []byte) error

type LiveServerMessage

type LiveServerMessage struct {
	// Sent in response to a `LiveClientSetup` message from the client.
	SetupComplete *LiveServerSetupComplete `json:"setupComplete,omitempty"`
	// Content generated by the model in response to client messages.
	ServerContent *LiveServerContent `json:"serverContent,omitempty"`
	// Request for the client to execute the `function_calls` and return the responses with
	// the matching `id`s.
	ToolCall *LiveServerToolCall `json:"toolCall,omitempty"`
	// Notification for the client that a previously issued `ToolCallMessage` with the specified
	// `id`s should have been not executed and should be cancelled.
	ToolCallCancellation *LiveServerToolCallCancellation `json:"toolCallCancellation,omitempty"`
	// Usage metadata about model response(s).
	UsageMetadata *UsageMetadata `json:"usageMetadata,omitempty"`
	// Server will disconnect soon.
	GoAway *LiveServerGoAway `json:"goAway,omitempty"`
	// Update of the session resumption state.
	SessionResumptionUpdate *LiveServerSessionResumptionUpdate `json:"sessionResumptionUpdate,omitempty"`
}

Response message for API call.

type LiveServerSessionResumptionUpdate added in v1.0.0

type LiveServerSessionResumptionUpdate struct {
	// New handle that represents state that can be resumed. Empty if `resumable`=false.
	NewHandle string `json:"newHandle,omitempty"`
	// True if session can be resumed at this point. It might be not possible to resume
	// session at some points. In that case we send update empty new_handle and resumable=false.
	// Example of such case could be model executing function calls or just generating.
	// Resuming session (using previous session token) in such state will result in some
	// data loss.
	Resumable bool `json:"resumable,omitempty"`
	// Index of last message sent by client that is included in state represented by this
	// SessionResumptionToken. Only sent when `SessionResumptionConfig.transparent` is set.
	// Presence of this index allows users to transparently reconnect and avoid issue of
	// losing some part of realtime audio input/video. If client wishes to temporarily disconnect
	// (for example as result of receiving GoAway) they can do it without losing state by
	// buffering messages sent since last `SessionResmumptionTokenUpdate`. This field will
	// enable them to limit buffering (avoid keeping all requests in RAM).
	// Note: This should not be used for when resuming a session at some time later -- in
	// those cases partial audio and video frames arelikely not needed.
	LastConsumedClientMessageIndex int64 `json:"lastConsumedClientMessageIndex,omitempty"`
}

Update of the session resumption state. Only sent if `session_resumption` was set in the connection config.

func (*LiveServerSessionResumptionUpdate) MarshalJSON added in v1.0.0

func (s *LiveServerSessionResumptionUpdate) MarshalJSON() ([]byte, error)

func (*LiveServerSessionResumptionUpdate) UnmarshalJSON added in v1.0.0

func (s *LiveServerSessionResumptionUpdate) UnmarshalJSON(data []byte) error

type LiveServerSetupComplete

type LiveServerSetupComplete struct {
}

Sent in response to a `LiveGenerateContentSetup` message from the client.

type LiveServerToolCall

type LiveServerToolCall struct {
	// The function call to be executed.
	FunctionCalls []*FunctionCall `json:"functionCalls,omitempty"`
}

Request for the client to execute the `function_calls` and return the responses with the matching `id`s.

type LiveServerToolCallCancellation

type LiveServerToolCallCancellation struct {
	// The IDs of the tool calls to be cancelled.
	IDs []string `json:"ids,omitempty"`
}

Notification for the client that a previously issued `ToolCallMessage` with the specified `id`s should have been not executed and should be cancelled. If there were side-effects to those tool calls, clients may attempt to undo the tool calls. This message occurs only in cases where the clients interrupt server turns.

type LiveToolResponseInput added in v1.0.0

type LiveToolResponseInput struct {
	// The response to the function calls.
	FunctionResponses []*FunctionResponse `json:"functionResponses,omitempty"`
}

Preview. LiveToolResponseInput is the input for [SendToolResponse].

type LogprobsResult

type LogprobsResult struct {
	// Length = total number of decoding steps. The chosen candidates may or may not be
	// in top_candidates.
	ChosenCandidates []*LogprobsResultCandidate `json:"chosenCandidates,omitempty"`
	// Length = total number of decoding steps.
	TopCandidates []*LogprobsResultTopCandidates `json:"topCandidates,omitempty"`
}

Logprobs Result

type LogprobsResultCandidate

type LogprobsResultCandidate struct {
	// The candidate's log probability.
	LogProbability float32 `json:"logProbability,omitempty"`
	// The candidate's token string value.
	Token string `json:"token,omitempty"`
	// The candidate's token ID value.
	TokenID int32 `json:"tokenId,omitempty"`
}

Candidate for the logprobs token and score.

type LogprobsResultTopCandidates

type LogprobsResultTopCandidates struct {
	// Sorted by log probability in descending order.
	Candidates []*LogprobsResultCandidate `json:"candidates,omitempty"`
}

Candidates with top log probabilities at each decoding step.

type MaskReferenceConfig

type MaskReferenceConfig struct {
	// Prompts the model to generate a mask instead of you needing to
	// provide one (unless MASK_MODE_USER_PROVIDED is used).
	MaskMode MaskReferenceMode `json:"maskMode,omitempty"`
	// A list of up to 5 class IDs to use for semantic segmentation.
	// Automatically creates an image mask based on specific objects.
	SegmentationClasses []int32 `json:"segmentationClasses,omitempty"`
	// Dilation percentage of the mask provided.
	// Float between 0 and 1.
	MaskDilation *float32 `json:"maskDilation,omitempty"`
}

Configuration for a Mask reference image.

type MaskReferenceImage

type MaskReferenceImage struct {
	// The reference image for the editing operation.
	ReferenceImage *Image `json:"referenceImage,omitempty"`
	// The ID of the reference image.
	ReferenceID int32 `json:"referenceId,omitempty"`

	// Configuration for the mask reference image.
	Config *MaskReferenceConfig `json:"config,omitempty"`
	// contains filtered or unexported fields
}

A mask reference image. This encapsulates either a mask image provided by the user and configs for the user provided mask, or only config parameters for the model to generate a mask. A mask image is an image whose non-zero values indicate where to edit the base image. If the user provides a mask image, the mask must be in the same dimensions as the raw image.

func NewMaskReferenceImage added in v0.5.0

func NewMaskReferenceImage(referenceImage *Image, referenceID int32, config *MaskReferenceConfig) *MaskReferenceImage

NewMaskReferenceImage creates a new MaskReferenceImage.

type MaskReferenceMode

type MaskReferenceMode string

Enum representing the mask mode of a mask reference image.

const (
	MaskReferenceModeMaskModeDefault      MaskReferenceMode = "MASK_MODE_DEFAULT"
	MaskReferenceModeMaskModeUserProvided MaskReferenceMode = "MASK_MODE_USER_PROVIDED"
	MaskReferenceModeMaskModeBackground   MaskReferenceMode = "MASK_MODE_BACKGROUND"
	MaskReferenceModeMaskModeForeground   MaskReferenceMode = "MASK_MODE_FOREGROUND"
	MaskReferenceModeMaskModeSemantic     MaskReferenceMode = "MASK_MODE_SEMANTIC"
)

type MediaModality added in v0.7.0

type MediaModality string

Server content modalities.

const (
	// The modality is unspecified.
	MediaModalityUnspecified MediaModality = "MODALITY_UNSPECIFIED"
	// Plain text.
	MediaModalityText MediaModality = "TEXT"
	// Images.
	MediaModalityImage MediaModality = "IMAGE"
	// Video.
	MediaModalityVideo MediaModality = "VIDEO"
	// Audio.
	MediaModalityAudio MediaModality = "AUDIO"
	// Document, e.g. PDF.
	MediaModalityDocument MediaModality = "DOCUMENT"
)

type MediaResolution

type MediaResolution string

The media resolution to use.

const (
	// Media resolution has not been set
	MediaResolutionUnspecified MediaResolution = "MEDIA_RESOLUTION_UNSPECIFIED"
	// Media resolution set to low (64 tokens).
	MediaResolutionLow MediaResolution = "MEDIA_RESOLUTION_LOW"
	// Media resolution set to medium (256 tokens).
	MediaResolutionMedium MediaResolution = "MEDIA_RESOLUTION_MEDIUM"
	// Media resolution set to high (zoomed reframing with 256 tokens).
	MediaResolutionHigh MediaResolution = "MEDIA_RESOLUTION_HIGH"
)

type Modality added in v0.1.0

type Modality string

Server content modalities.

const (
	// The modality is unspecified.
	ModalityUnspecified Modality = "MODALITY_UNSPECIFIED"
	// Indicates the model should return text
	ModalityText Modality = "TEXT"
	// Indicates the model should return images.
	ModalityImage Modality = "IMAGE"
	// Indicates the model should return images.
	ModalityAudio Modality = "AUDIO"
)

type ModalityTokenCount added in v0.7.0

type ModalityTokenCount struct {
	// The modality associated with this token count.
	Modality MediaModality `json:"modality,omitempty"`
	// Number of tokens.
	TokenCount int32 `json:"tokenCount,omitempty"`
}

Represents token counting info for a single modality.

type Mode

type Mode string

The mode of the predictor to be used in dynamic retrieval.

const (
	// Always trigger retrieval.
	ModeUnspecified Mode = "MODE_UNSPECIFIED"
	// Run retrieval only when system decides it is necessary.
	ModeDynamic Mode = "MODE_DYNAMIC"
)

type Model added in v0.4.0

type Model struct {
	// Resource name of the model.
	Name string `json:"name,omitempty"`
	// Display name of the model.
	DisplayName string `json:"displayName,omitempty"`
	// Description of the model.
	Description string `json:"description,omitempty"`
	// Version ID of the model. A new version is committed when a new
	// model version is uploaded or trained under an existing model ID. The
	// version ID is an auto-incrementing decimal number in string
	// representation.
	Version string `json:"version,omitempty"`
	// List of deployed models created from this base model. Note that a
	// model could have been deployed to endpoints in different locations.
	Endpoints []*Endpoint `json:"endpoints,omitempty"`
	// Labels with user-defined metadata to organize your models.
	Labels map[string]string `json:"labels,omitempty"`
	// Information about the tuned model from the base model.
	TunedModelInfo *TunedModelInfo `json:"tunedModelInfo,omitempty"`
	// The maximum number of input tokens that the model can handle.
	InputTokenLimit int32 `json:"inputTokenLimit,omitempty"`
	// The maximum number of output tokens that the model can generate.
	OutputTokenLimit int32 `json:"outputTokenLimit,omitempty"`
	// List of actions that are supported by the model.
	SupportedActions []string `json:"supportedActions,omitempty"`
}

A trained machine learning model.

type ModelSelectionConfig added in v1.1.0

type ModelSelectionConfig struct {
	// Options for feature selection preference.
	FeatureSelectionPreference FeatureSelectionPreference `json:"featureSelectionPreference,omitempty"`
}

Config for model selection.

type Models

type Models struct {
	// contains filtered or unexported fields
}

Models provides methods for interacting with the available language models. You don't need to initiate this struct. Create a client instance via NewClient, and then access Models through client.Models field.

func (Models) All added in v0.5.0

func (m Models) All(ctx context.Context) iter.Seq2[*Model, error]

All retrieves all models resources.

This method handles pagination internally, making multiple API calls as needed to fetch all entries. It returns an iterator that yields each cached content entry one by one. You do not need to manage pagination tokens or make multiple calls to retrieve all data.

func (Models) ComputeTokens added in v0.1.0

func (m Models) ComputeTokens(ctx context.Context, model string, contents []*Content, config *ComputeTokensConfig) (*ComputeTokensResponse, error)

ComputeTokens computes the number of tokens for the provided contents.

func (Models) CountTokens added in v0.1.0

func (m Models) CountTokens(ctx context.Context, model string, contents []*Content, config *CountTokensConfig) (*CountTokensResponse, error)

CountTokens counts the number of tokens in the provided contents.

func (Models) Delete added in v0.4.0

func (m Models) Delete(ctx context.Context, model string, config *DeleteModelConfig) (*DeleteModelResponse, error)

Delete deletes a specific model resource by its name.

func (Models) EditImage added in v0.5.0

func (m Models) EditImage(ctx context.Context, model, prompt string, referenceImages []ReferenceImage, config *EditImageConfig) (*EditImageResponse, error)

EditImage edits an image based on the provided model, prompt, reference images, and configuration.

func (Models) EmbedContent added in v0.5.0

func (m Models) EmbedContent(ctx context.Context, model string, contents []*Content, config *EmbedContentConfig) (*EmbedContentResponse, error)

EmbedContent generates embeddings for the provided contents using the specified model.

func (Models) GenerateContent

func (m Models) GenerateContent(ctx context.Context, model string, contents []*Content, config *GenerateContentConfig) (*GenerateContentResponse, error)

GenerateContent generates content based on the provided model, contents, and configuration.

Example (CodeExecution_geminiapi)

This example shows how to call the GenerateContent method with code execution to Gemini API.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx,
		"gemini-2.0-flash",
		genai.Text("What is the sum of the first 50 prime numbers? Generate and run code for the calculation, and make sure you get all 50."),
		&genai.GenerateContentConfig{
			Tools: []*genai.Tool{
				{CodeExecution: &genai.ToolCodeExecution{}},
			},
		},
	)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (CodeExecution_vertexai)

This example shows how to call the GenerateContent method with code execution to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx,
		"gemini-2.0-flash",
		genai.Text("What is the sum of the first 50 prime numbers? Generate and run code for the calculation, and make sure you get all 50."),
		&genai.GenerateContentConfig{
			Tools: []*genai.Tool{
				{CodeExecution: &genai.ToolCodeExecution{}},
			},
		},
	)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (Config_geminiapi)

This example shows how to call the GenerateContent method with GenerateContentConfig to Gemini API.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx,
		"gemini-2.0-flash",
		genai.Text("Tell me about New York?"),
		&genai.GenerateContentConfig{
			Temperature:      genai.Ptr[float32](0.5),
			TopP:             genai.Ptr[float32](0.5),
			TopK:             genai.Ptr[float32](2.0),
			ResponseMIMEType: "application/json",
			StopSequences:    []string{"\n"},
			CandidateCount:   2,
			Seed:             genai.Ptr[int32](42),
			MaxOutputTokens:  128,
			PresencePenalty:  genai.Ptr[float32](0.5),
			FrequencyPenalty: genai.Ptr[float32](0.5),
		},
	)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (Config_vertexai)

This example shows how to call the GenerateContent method with GenerateContentConfig to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx,
		"gemini-2.0-flash",
		genai.Text("Tell me about New York?"),
		&genai.GenerateContentConfig{
			Temperature:      genai.Ptr[float32](0.5),
			TopP:             genai.Ptr[float32](0.5),
			TopK:             genai.Ptr[float32](2.0),
			ResponseMIMEType: "application/json",
			StopSequences:    []string{"\n"},
			CandidateCount:   2,
			Seed:             genai.Ptr[int32](42),
			MaxOutputTokens:  128,
			PresencePenalty:  genai.Ptr[float32](0.5),
			FrequencyPenalty: genai.Ptr[float32](0.5),
		},
	)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (GcsURI_vertexai)

This example shows how to call the GenerateContent method with GCS URI to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	parts := []*genai.Part{
		{Text: "What's this video about?"},
		{FileData: &genai.FileData{FileURI: "gs://cloud-samples-data/video/animals.mp4", MIMEType: "video/mp4"}},
	}
	contents := []*genai.Content{{Parts: parts}}

	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (GoogleSearchRetrieval_geminiapi)

This example shows how to call the GenerateContent method with Google Search Retrieval to Gemini API.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx,
		"gemini-2.0-flash",
		genai.Text("Tell me about New York?"),
		&genai.GenerateContentConfig{
			Tools: []*genai.Tool{
				{GoogleSearchRetrieval: &genai.GoogleSearchRetrieval{}},
			},
		},
	)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (GoogleSearchRetrieval_vertexai)

This example shows how to call the GenerateContent method with Google Search Retrieval to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	parts := []*genai.Part{{Text: "Tell me about New York?"}}
	contents := []*genai.Content{{Parts: parts}}

	result, err := client.Models.GenerateContent(ctx,
		"gemini-2.0-flash",
		contents,
		&genai.GenerateContentConfig{
			Tools: []*genai.Tool{
				{GoogleSearchRetrieval: &genai.GoogleSearchRetrieval{}},
			},
		},
	)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (HttpURL_vertexai)

This example shows how to call the GenerateContent method with HTTP URL to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	parts := []*genai.Part{
		{Text: "What's this picture about?"},
		{FileData: &genai.FileData{FileURI: "https://storage.googleapis.com/cloud-samples-data/generative-ai/image/scones.jpg", MIMEType: "image/jpeg"}},
	}
	contents := []*genai.Content{{Parts: parts}}

	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (InlineAudio_geminiapi)

This example shows how to call the GenerateContent method with a inline audio file to Gemini API.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	resp, err := http.Get("your audio url")
	if err != nil {
		fmt.Println("Error fetching image:", err)
		return
	}
	defer resp.Body.Close()
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	parts := []*genai.Part{
		{Text: "What's this music about?"},
		{InlineData: &genai.Blob{Data: data, MIMEType: "audio/mp3"}},
	}
	contents := []*genai.Content{{Parts: parts}}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (InlineAudio_vertexai)

This example shows how to call the GenerateContent method with a inline audio file to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Read the audio file.
	resp, err := http.Get("your audio url")
	if err != nil {
		fmt.Println("Error fetching image:", err)
		return
	}
	defer resp.Body.Close()
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	parts := []*genai.Part{
		{Text: "What's this music about?"},
		{InlineData: &genai.Blob{Data: data, MIMEType: "audio/mp3"}},
	}
	contents := []*genai.Content{{Parts: parts}}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (InlineImage_geminiapi)

This example shows how to call the GenerateContent method with inline image to Gemini API.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Read the image data from a url.
	resp, err := http.Get("https://storage.googleapis.com/cloud-samples-data/generative-ai/image/scones.jpg")
	if err != nil {
		fmt.Println("Error fetching image:", err)
		return
	}
	defer resp.Body.Close()
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	parts := []*genai.Part{
		{Text: "What's this image about?"},
		{InlineData: &genai.Blob{Data: data, MIMEType: "image/jpeg"}},
	}
	contents := []*genai.Content{{Parts: parts}}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (InlineImage_vertexai)

This example shows how to call the GenerateContent method with inline image to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Read the image data from a url.
	resp, err := http.Get("https://storage.googleapis.com/cloud-samples-data/generative-ai/image/scones.jpg")
	if err != nil {
		fmt.Println("Error fetching image:", err)
		return
	}
	defer resp.Body.Close()
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	parts := []*genai.Part{
		{Text: "What's this image about?"},
		{InlineData: &genai.Blob{Data: data, MIMEType: "image/jpeg"}},
	}
	contents := []*genai.Content{{Parts: parts}}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (InlinePDF_geminiapi)

This example shows how to call the GenerateContent method with a inline pdf file to Gemini API.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Read the pdf file.
	resp, err := http.Get("your pdf url")
	if err != nil {
		fmt.Println("Error fetching image:", err)
		return
	}
	defer resp.Body.Close()
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	parts := []*genai.Part{
		{Text: "What's this pdf about?"},
		{InlineData: &genai.Blob{Data: data, MIMEType: "application/pdf"}},
	}
	contents := []*genai.Content{{Parts: parts}}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (InlinePDF_vertexai)

This example shows how to call the GenerateContent method with a inline pdf file to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Read the pdf file.
	resp, err := http.Get("your pdf url")
	if err != nil {
		fmt.Println("Error fetching image:", err)
		return
	}
	defer resp.Body.Close()
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	parts := []*genai.Part{
		{Text: "What's this pdf about?"},
		{InlineData: &genai.Blob{Data: data, MIMEType: "application/pdf"}},
	}
	contents := []*genai.Content{{Parts: parts}}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (InlineVideo_geminiapi)

This example shows how to call the GenerateContent method with a inline video file to Gemini API.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Read the video file.
	resp, err := http.Get("your video url")
	if err != nil {
		fmt.Println("Error fetching image:", err)
		return
	}
	defer resp.Body.Close()
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	parts := []*genai.Part{
		{Text: "What's this video about?"},
		{InlineData: &genai.Blob{Data: data, MIMEType: "video/mp4"}},
	}
	contents := []*genai.Content{{Parts: parts}}

	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (InlineVideo_vertexai)

This example shows how to call the GenerateContent method with a inline video file to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Read the video file.
	resp, err := http.Get("your video url")
	if err != nil {
		fmt.Println("Error fetching image:", err)
		return
	}
	defer resp.Body.Close()
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	parts := []*genai.Part{
		{Text: "What's this video about?"},
		{InlineData: &genai.Blob{Data: data, MIMEType: "video/mp4"}},
	}
	contents := []*genai.Content{{Parts: parts}}

	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (SystemInstruction_geminiapi)

This example shows how to call the GenerateContent method with system instruction to Gemini API.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx,
		"gemini-2.0-flash",
		genai.Text("Tell me about New York?"),
		&genai.GenerateContentConfig{
			SystemInstruction: &genai.Content{Parts: []*genai.Part{{Text: "You are a helpful assistant."}}},
		},
	)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (SystemInstruction_vertexai)

This example shows how to call the GenerateContent method with system instruction to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx,
		"gemini-2.0-flash",
		genai.Text("Tell me about New York?"),
		&genai.GenerateContentConfig{
			SystemInstruction: &genai.Content{Parts: []*genai.Part{{Text: "You are a helpful assistant."}}},
		},
	)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (Text_geminiapi)

This example shows how to call the GenerateContent method with a simple text to Gemini API.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", genai.Text("Tell me about New York?"), nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (Text_vertexai)

This example shows how to call the GenerateContent method with a simple text to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", genai.Text("Tell me about New York?"), nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (Texts_geminiapi)

This example shows how to call the GenerateContent method with multiple texts to Gemini API.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	parts := []*genai.Part{
		{Text: "Tell me about New York?"},
		{Text: "And how about San Francison?"},
	}
	contents := []*genai.Content{{Parts: parts}}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (Texts_vertexai)

This example shows how to call the GenerateContent method with multiple texts to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	parts := []*genai.Part{
		{Text: "Tell me about New York?"},
		{Text: "And how about San Francison?"},
	}
	contents := []*genai.Content{{Parts: parts}}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", contents, nil)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

Example (ThirdPartyModel_vertexai)

This example shows how to call the GenerateContent method with third party model to Vertex AI.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContent method.
	result, err := client.Models.GenerateContent(ctx,
		"meta/llama-3.2-90b-vision-instruct-maas",
		genai.Text("Tell me about New York?"),
		nil,
	)
	if err != nil {
		log.Fatal(err)
	}
	debugPrint(result)
}

func debugPrint[T any](r *T) {

	response, err := json.MarshalIndent(*r, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(response))
}
Output:

func (Models) GenerateContentStream

func (m Models) GenerateContentStream(ctx context.Context, model string, contents []*Content, config *GenerateContentConfig) iter.Seq2[*GenerateContentResponse, error]

GenerateContentStream generates a stream of content based on the provided model, contents, and configuration.

Example (Text_geminiapi)

This example shows how to call the GenerateContentStream method with a simple text to Gemini API.

package main

import (
	"context"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your Google API key
const apiKey = "your-api-key"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		APIKey:  apiKey,
		Backend: genai.BackendGeminiAPI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContentStream method.
	for result, err := range client.Models.GenerateContentStream(
		ctx,
		"gemini-2.0-flash",
		genai.Text("Give me top 3 indoor kids friendly ideas."),
		nil,
	) {
		if err != nil {
			log.Fatal(err)
		}
		fmt.Print(result.Candidates[0].Content.Parts[0].Text)
	}
}
Output:

Example (Text_vertexai)

This example shows how to call the GenerateContentStream method with a simple text to Vertex AI.

package main

import (
	"context"
	"fmt"
	"log"

	"google.golang.org/genai"
)

// Your GCP project
const project = "your-project"

// A GCP location like "us-central1"
const location = "some-gcp-location"

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		Project:  project,
		Location: location,
		Backend:  genai.BackendVertexAI,
	})
	if err != nil {
		log.Fatal(err)
	}

	// Call the GenerateContentStream method.
	for result, err := range client.Models.GenerateContentStream(
		ctx,
		"gemini-2.0-flash",
		genai.Text("Give me top 3 indoor kids friendly ideas."),
		nil,
	) {
		if err != nil {
			log.Fatal(err)
		}
		fmt.Print(result.Candidates[0].Content.Parts[0].Text)
	}
}
Output:

func (Models) GenerateImages added in v0.1.0

func (m Models) GenerateImages(ctx context.Context, model string, prompt string, config *GenerateImagesConfig) (*GenerateImagesResponse, error)

GenerateImages generates images based on the provided model, prompt, and configuration.

func (Models) GenerateVideos added in v0.7.0

func (m Models) GenerateVideos(ctx context.Context, model string, prompt string, image *Image, config *GenerateVideosConfig) (*GenerateVideosOperation, error)

GenerateVideos creates a long-running video generation operation.

func (Models) Get added in v0.4.0

func (m Models) Get(ctx context.Context, model string, config *GetModelConfig) (*Model, error)

Get retrieves a specific model resource by its name.

func (Models) List added in v0.5.0

func (m Models) List(ctx context.Context, config *ListModelsConfig) (Page[Model], error)

List retrieves a paginated list of models resources.

func (Models) Update added in v0.4.0

func (m Models) Update(ctx context.Context, model string, config *UpdateModelConfig) (*Model, error)

Update updates a specific model resource.

func (Models) UpscaleImage added in v0.4.0

func (m Models) UpscaleImage(ctx context.Context, model string, image *Image, upscaleFactor string, config *UpscaleImageConfig) (*UpscaleImageResponse, error)

UpscaleImage upscales an image using the specified model, image, upscale factor, and configuration.

type Operations added in v0.7.0

type Operations struct {
	// contains filtered or unexported fields
}

Operations provides methods for managing the long-running operations. You don't need to initiate this struct. Create a client instance via NewClient, and then access Operations through client.Operations field.

func (Operations) GetVideosOperation added in v0.7.0

func (m Operations) GetVideosOperation(ctx context.Context, operation *GenerateVideosOperation, config *GetOperationConfig) (*GenerateVideosOperation, error)

GetVideosOperation retrieves the status and result of a long-running video generation operation.

If the operation is still in progress, the returned GenerateVideosOperation will have Done set to false. If the operation has completed successfully, Done will be true, and the Result field will contain the result. If the operation failed, Done will be true, and the Error field will be populated.

type Outcome

type Outcome string

Outcome of the code execution.

const (
	// Unspecified status. This value should not be used.
	OutcomeUnspecified Outcome = "OUTCOME_UNSPECIFIED"
	// Code execution completed successfully.
	OutcomeOK Outcome = "OUTCOME_OK"
	// Code execution finished but with a failure. `stderr` should contain the reason.
	OutcomeFailed Outcome = "OUTCOME_FAILED"
	// Code execution ran for too long, and was cancelled. There may or may not be a partial
	// output present.
	OutcomeDeadlineExceeded Outcome = "OUTCOME_DEADLINE_EXCEEDED"
)

type Page added in v0.4.0

type Page[T any] struct {
	Name          string // The name of the resource.
	Items         []*T   // The items in the current page.
	NextPageToken string // The token to use to retrieve the next page of results.
	// contains filtered or unexported fields
}

Page represents a page of results from a paginated API call. It contains a slice of items and information about the next page.

func (Page[T]) Next added in v0.4.0

func (p Page[T]) Next(ctx context.Context) (Page[T], error)

Next retrieves the next page of results.

If there are no more pages, PageDone is returned. Otherwise, a new Page struct containing the next set of results is returned. Any other errors encountered during retrieval will also be returned.

type Part

type Part struct {
	// Metadata for a given video.
	VideoMetadata *VideoMetadata `json:"videoMetadata,omitempty"`
	// Indicates if the part is thought from the model.
	Thought bool `json:"thought,omitempty"`
	// Optional. Result of executing the [ExecutableCode].
	CodeExecutionResult *CodeExecutionResult `json:"codeExecutionResult,omitempty"`
	// Optional. Code generated by the model that is meant to be executed.
	ExecutableCode *ExecutableCode `json:"executableCode,omitempty"`
	// Optional. URI based data.
	FileData *FileData `json:"fileData,omitempty"`
	// Optional. A predicted [FunctionCall] returned from the model that contains a string
	// representing the [FunctionDeclaration.Name] with the parameters and their values.
	FunctionCall *FunctionCall `json:"functionCall,omitempty"`
	// Optional. The result output of a [FunctionCall] that contains a string representing
	// the [FunctionDeclaration.Name] and a structured JSON object containing any output
	// from the function call. It is used as context to the model.
	FunctionResponse *FunctionResponse `json:"functionResponse,omitempty"`
	// Optional. Inlined bytes data.
	InlineData *Blob `json:"inlineData,omitempty"`
	// Optional. Text part (can be code).
	Text string `json:"text,omitempty"`
}

A datatype containing media content. Exactly one field within a Part should be set, representing the specific type of content being conveyed. Using multiple fields within the same `Part` instance is considered invalid.

func NewPartFromBytes added in v0.1.0

func NewPartFromBytes(data []byte, mimeType string) *Part

NewPartFromBytes builds a Part from a given byte array and mime type.

func NewPartFromCodeExecutionResult added in v0.1.0

func NewPartFromCodeExecutionResult(outcome Outcome, output string) *Part

NewPartFromCodeExecutionResult builds a CodeExecutionResult Part from the given Outcome and std output.

func NewPartFromExecutableCode added in v0.1.0

func NewPartFromExecutableCode(code string, language Language) *Part

NewPartFromExecutableCode builds a ExecutableCode Part from a single piece of source code in the given Language.

func NewPartFromFunctionCall added in v0.1.0

func NewPartFromFunctionCall(name string, args map[string]any) *Part

NewPartFromFunctionCall builds a FunctionCall Part from the given function name and args.

func NewPartFromFunctionResponse added in v0.1.0

func NewPartFromFunctionResponse(name string, response map[string]any) *Part

NewPartFromFunctionResponse builds a FunctionResponse Part from the given function name and response.

func NewPartFromText added in v0.1.0

func NewPartFromText(text string) *Part

NewPartFromText builds a Part from a given text.

func NewPartFromURI added in v0.1.0

func NewPartFromURI(fileURI, mimeType string) *Part

NewPartFromURI builds a Part from a given file URI and mime type.

type PersonGeneration added in v0.1.0

type PersonGeneration string

Enum that controls the generation of people.

const (
	PersonGenerationDontAllow  PersonGeneration = "DONT_ALLOW"
	PersonGenerationAllowAdult PersonGeneration = "ALLOW_ADULT"
	PersonGenerationAllowAll   PersonGeneration = "ALLOW_ALL"
)

type PrebuiltVoiceConfig

type PrebuiltVoiceConfig struct {
	// The name of the prebuilt voice to use.
	VoiceName string `json:"voiceName,omitempty"`
}

The configuration for the prebuilt speaker to use.

type RAGRetrievalConfig added in v1.1.0

type RAGRetrievalConfig struct {
	// Optional. Config for filters.
	Filter *RAGRetrievalConfigFilter `json:"filter,omitempty"`
	// Optional. Config for Hybrid Search.
	HybridSearch *RAGRetrievalConfigHybridSearch `json:"hybridSearch,omitempty"`
	// Optional. Config for ranking and reranking.
	Ranking *RAGRetrievalConfigRanking `json:"ranking,omitempty"`
	// Optional. The number of contexts to retrieve.
	TopK *int32 `json:"topK,omitempty"`
}

Specifies the context retrieval config.

type RAGRetrievalConfigFilter added in v1.1.0

type RAGRetrievalConfigFilter struct {
	// Optional. String for metadata filtering.
	MetadataFilter string `json:"metadataFilter,omitempty"`
	// Optional. Only returns contexts with vector distance smaller than the threshold.
	VectorDistanceThreshold *float64 `json:"vectorDistanceThreshold,omitempty"`
	// Optional. Only returns contexts with vector similarity larger than the threshold.
	VectorSimilarityThreshold *float64 `json:"vectorSimilarityThreshold,omitempty"`
}

Config for filters.

type RAGRetrievalConfigHybridSearch added in v1.1.0

type RAGRetrievalConfigHybridSearch struct {
	// Optional. Alpha value controls the weight between dense and sparse vector search
	// results. The range is [0, 1], while 0 means sparse vector search only and 1 means
	// dense vector search only. The default value is 0.5 which balances sparse and dense
	// vector search equally.
	Alpha *float32 `json:"alpha,omitempty"`
}

Config for Hybrid Search.

type RAGRetrievalConfigRanking added in v1.1.0

type RAGRetrievalConfigRanking struct {
	// Optional. Config for LlmRanker.
	LlmRanker *RAGRetrievalConfigRankingLlmRanker `json:"llmRanker,omitempty"`
	// Optional. Config for Rank Service.
	RankService *RAGRetrievalConfigRankingRankService `json:"rankService,omitempty"`
}

Config for ranking and reranking.

type RAGRetrievalConfigRankingLlmRanker added in v1.1.0

type RAGRetrievalConfigRankingLlmRanker struct {
	// Optional. The model name used for ranking. Format: `gemini-1.5-pro`
	ModelName string `json:"modelName,omitempty"`
}

Config for LlmRanker.

type RAGRetrievalConfigRankingRankService added in v1.1.0

type RAGRetrievalConfigRankingRankService struct {
	// Optional. The model name of the rank service. Format: `semantic-ranker-512@latest`
	ModelName string `json:"modelName,omitempty"`
}

Config for Rank Service.

type RawReferenceImage

type RawReferenceImage struct {
	// The reference image for the editing operation.
	ReferenceImage *Image `json:"referenceImage,omitempty"`
	// The ID of the reference image.
	ReferenceID int32 `json:"referenceId,omitempty"`
	// contains filtered or unexported fields
}

A raw reference image. A raw reference image represents the base image to edit, provided by the user. It can optionally be provided in addition to a mask reference image or a style reference image.

func NewRawReferenceImage added in v0.5.0

func NewRawReferenceImage(referenceImage *Image, referenceID int32) *RawReferenceImage

NewRawReferenceImage creates a new RawReferenceImage.

type RealtimeInputConfig added in v1.0.0

type RealtimeInputConfig struct {
	// If not set, automatic activity detection is enabled by default. If automatic voice
	// detection is disabled, the client must send activity signals.
	AutomaticActivityDetection *AutomaticActivityDetection `json:"automaticActivityDetection,omitempty"`
	// Defines what effect activity has.
	ActivityHandling ActivityHandling `json:"activityHandling,omitempty"`
	// Defines which input is included in the user's turn.
	TurnCoverage TurnCoverage `json:"turnCoverage,omitempty"`
}

Marks the end of user activity. This can only be sent if automatic (i.e. server-side) activity detection is disabled.

type ReferenceImage added in v0.5.0

type ReferenceImage interface {
	// contains filtered or unexported methods
}

ReferenceImage is an interface that represents a generic reference image.

You can create instances that implement this interface using the following constructor functions:

  • NewRawReferenceImage
  • NewMaskReferenceImage
  • NewControlReferenceImage
  • NewStyleReferenceImage
  • NewSubjectReferenceImage
  • ...

type Retrieval

type Retrieval struct {
	// Optional. Deprecated. This option is no longer supported.
	DisableAttribution bool `json:"disableAttribution,omitempty"`
	// Set to use data source powered by Vertex AI Search.
	VertexAISearch *VertexAISearch `json:"vertexAiSearch,omitempty"`
	// Set to use data source powered by Vertex RAG store. User data is uploaded via the
	// VertexRAGDataService.
	VertexRAGStore *VertexRAGStore `json:"vertexRagStore,omitempty"`
}

Defines a retrieval tool that model can call to access external knowledge.

type RetrievalMetadata

type RetrievalMetadata struct {
	// Optional. Score indicating how likely information from Google Search could help answer
	// the prompt. The score is in the range `[0, 1]`, where 0 is the least likely and 1
	// is the most likely. This score is only populated when Google Search grounding and
	// dynamic retrieval is enabled. It will be compared to the threshold to determine whether
	// to trigger Google Search.
	GoogleSearchDynamicRetrievalScore float32 `json:"googleSearchDynamicRetrievalScore,omitempty"`
}

Metadata related to retrieval in the grounding flow.

type Role added in v1.0.0

type Role string

type SafetyAttributes added in v0.6.0

type SafetyAttributes struct {
	// List of RAI categories.
	Categories []string `json:"categories,omitempty"`
	// List of scores of each categories.
	Scores []float32 `json:"scores,omitempty"`
	// Internal use only.
	ContentType string `json:"contentType,omitempty"`
}

Safety attributes of a GeneratedImage or the user-provided prompt.

type SafetyFilterLevel added in v0.1.0

type SafetyFilterLevel string

Enum that controls the safety filter level for objectionable content.

const (
	SafetyFilterLevelBlockLowAndAbove    SafetyFilterLevel = "BLOCK_LOW_AND_ABOVE"
	SafetyFilterLevelBlockMediumAndAbove SafetyFilterLevel = "BLOCK_MEDIUM_AND_ABOVE"
	SafetyFilterLevelBlockOnlyHigh       SafetyFilterLevel = "BLOCK_ONLY_HIGH"
	SafetyFilterLevelBlockNone           SafetyFilterLevel = "BLOCK_NONE"
)

type SafetyRating

type SafetyRating struct {
	// Output only. Indicates whether the content was filtered out because of this rating.
	Blocked bool `json:"blocked,omitempty"`
	// Output only. Harm category.
	Category HarmCategory `json:"category,omitempty"`
	// Output only. Harm probability levels in the content.
	Probability HarmProbability `json:"probability,omitempty"`
	// Output only. Harm probability score.
	ProbabilityScore float32 `json:"probabilityScore,omitempty"`
	// Output only. Harm severity levels in the content.
	Severity HarmSeverity `json:"severity,omitempty"`
	// Output only. Harm severity score.
	SeverityScore float32 `json:"severityScore,omitempty"`
}

Safety rating corresponding to the generated content.

type SafetySetting

type SafetySetting struct {
	// Determines if the harm block method uses probability or probability
	// and severity scores.
	Method HarmBlockMethod `json:"method,omitempty"`
	// Required. Harm category.
	Category HarmCategory `json:"category,omitempty"`
	// Required. The harm block threshold.
	Threshold HarmBlockThreshold `json:"threshold,omitempty"`
}

Safety settings.

type Schema

type Schema struct {
	// Optional. Example of the object. Will only populated when the object is the root.
	Example any `json:"example,omitempty"`
	// Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
	Pattern string `json:"pattern,omitempty"`
	// Optional. Default value of the data.
	Default any `json:"default,omitempty"`
	// Optional. Maximum length of the Type.STRING
	MaxLength *int64 `json:"maxLength,omitempty"`
	// Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
	MinLength *int64 `json:"minLength,omitempty"`
	// Optional. Minimum number of the properties for Type.OBJECT.
	MinProperties *int64 `json:"minProperties,omitempty"`
	// Optional. Maximum number of the properties for Type.OBJECT.
	MaxProperties *int64 `json:"maxProperties,omitempty"`
	// Optional. The value should be validated against any (one or more) of the subschemas
	// in the list.
	AnyOf []*Schema `json:"anyOf,omitempty"`
	// Optional. The description of the data.
	Description string `json:"description,omitempty"`
	// Optional. Possible values of the element of primitive type with enum format. Examples:
	// 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH",
	// "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum,
	// enum:["101", "201", "301"]}
	Enum []string `json:"enum,omitempty"`
	// Optional. The format of the data. Supported formats: for NUMBER type: "float", "double"
	// for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
	Format string `json:"format,omitempty"`
	// Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
	Items *Schema `json:"items,omitempty"`
	// Optional. Maximum number of the elements for Type.ARRAY.
	MaxItems *int64 `json:"maxItems,omitempty"`
	// Optional. Maximum value of the Type.INTEGER and Type.NUMBER
	Maximum *float64 `json:"maximum,omitempty"`
	// Optional. Minimum number of the elements for Type.ARRAY.
	MinItems *int64 `json:"minItems,omitempty"`
	// Optional. Minimum value of the Type.INTEGER and Type.NUMBER.
	Minimum *float64 `json:"minimum,omitempty"`
	// Optional. Indicates if the value may be null.
	Nullable *bool `json:"nullable,omitempty"`
	// Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
	Properties map[string]*Schema `json:"properties,omitempty"`
	// Optional. The order of the properties. Not a standard field in open API spec. Only
	// used to support the order of the properties.
	PropertyOrdering []string `json:"propertyOrdering,omitempty"`
	// Optional. Required properties of Type.OBJECT.
	Required []string `json:"required,omitempty"`
	// Optional. The title of the Schema.
	Title string `json:"title,omitempty"`
	// Optional. The type of the data.
	Type Type `json:"type,omitempty"`
}

Schema that defines the format of input and output data. Represents a select subset of an OpenAPI 3.0 schema object. You can find more details and examples at https://spec.openapis.org/oas/v3.0.3.html#schema-object

func (*Schema) MarshalJSON added in v0.6.0

func (s *Schema) MarshalJSON() ([]byte, error)

func (*Schema) UnmarshalJSON added in v0.6.0

func (s *Schema) UnmarshalJSON(data []byte) error

type SearchEntryPoint

type SearchEntryPoint struct {
	// Optional. Web content snippet that can be embedded in a web page or an app webview.
	RenderedContent string `json:"renderedContent,omitempty"`
	// Optional. Base64 encoded JSON representing array of tuple.
	SDKBlob []byte `json:"sdkBlob,omitempty"`
}

Google search entry point.

type Segment

type Segment struct {
	// Output only. End index in the given Part, measured in bytes. Offset from the start
	// of the Part, exclusive, starting at zero.
	EndIndex int32 `json:"endIndex,omitempty"`
	// Output only. The index of a Part object within its parent Content object.
	PartIndex int32 `json:"partIndex,omitempty"`
	// Output only. Start index in the given Part, measured in bytes. Offset from the start
	// of the Part, inclusive, starting at zero.
	StartIndex int32 `json:"startIndex,omitempty"`
	// Output only. The text corresponding to the segment from the response.
	Text string `json:"text,omitempty"`
}

Segment of the content.

type Session

type Session struct {
	// contains filtered or unexported fields
}

Preview. Session is a realtime connection to the API. The live module is experimental.

func (*Session) Close

func (s *Session) Close() error

Preview. Close terminates the connection. The live module is experimental.

func (*Session) Receive

func (s *Session) Receive() (*LiveServerMessage, error)

Preview. Receive reads a LiveServerMessage from the connection. It returns the received message or an error if reading or unmarshalling fails. The live module is experimental.

func (*Session) SendClientContent added in v1.0.0

func (s *Session) SendClientContent(input LiveClientContentInput) error

Preview. SendClientContent transmits a LiveClientContent over the established connection. It returns an error if sending the message fails. The live module is experimental.

func (*Session) SendRealtimeInput added in v1.0.0

func (s *Session) SendRealtimeInput(input LiveRealtimeInput) error

Preview. SendRealtimeInput transmits a LiveClientRealtimeInput over the established connection. It returns an error if sending the message fails. The live module is experimental.

func (*Session) SendToolResponse added in v1.0.0

func (s *Session) SendToolResponse(input LiveToolResponseInput) error

Preview. SendToolResponse transmits a LiveClientToolResponse over the established connection. It returns an error if sending the message fails. The live module is experimental.

type SessionResumptionConfig added in v1.0.0

type SessionResumptionConfig struct {
	// Session resumption handle of previous session (session to restore).
	// If not present new session will be started.
	Handle string `json:"handle,omitempty"`
	// If set the server will send `last_consumed_client_message_index` in the `session_resumption_update`
	// messages to allow for transparent reconnections.
	Transparent bool `json:"transparent,omitempty"`
}

Configuration of session resumption mechanism. Included in `LiveConnectConfig.session_resumption`. If included server will send `LiveServerSessionResumptionUpdate` messages.

type SlidingWindow added in v1.0.0

type SlidingWindow struct {
	// Session reduction target -- how many tokens we should keep. Window shortening operation
	// has some latency costs, so we should avoid running it on every turn. Should be <
	// trigger_tokens. If not set, trigger_tokens/2 is assumed.
	TargetTokens *int64 `json:"targetTokens,omitempty"`
}

Context window will be truncated by keeping only suffix of it. Context window will always be cut at start of USER role turn. System instructions and `BidiGenerateContentSetup.prefix_turns` will not be subject to the sliding window mechanism, they will always stay at the beginning of context window.

func (*SlidingWindow) MarshalJSON added in v1.0.0

func (s *SlidingWindow) MarshalJSON() ([]byte, error)

func (*SlidingWindow) UnmarshalJSON added in v1.0.0

func (s *SlidingWindow) UnmarshalJSON(data []byte) error

type SpeechConfig

type SpeechConfig struct {
	// The configuration for the speaker to use.
	VoiceConfig *VoiceConfig `json:"voiceConfig,omitempty"`
	// Language code (ISO 639. e.g. en-US) for the speech synthesization.
	// Only available for Live API.
	LanguageCode string `json:"languageCode,omitempty"`
}

The speech generation configuration.

type StartSensitivity added in v1.0.0

type StartSensitivity string

Start of speech sensitivity.

const (
	// The default is START_SENSITIVITY_LOW.
	StartSensitivityUnspecified StartSensitivity = "START_SENSITIVITY_UNSPECIFIED"
	// Automatic detection will detect the start of speech more often.
	StartSensitivityHigh StartSensitivity = "START_SENSITIVITY_HIGH"
	// Automatic detection will detect the start of speech less often.
	StartSensitivityLow StartSensitivity = "START_SENSITIVITY_LOW"
)

type StyleReferenceConfig

type StyleReferenceConfig struct {
	// A text description of the style to use for the generated image.
	StyleDescription string `json:"styleDescription,omitempty"`
}

Configuration for a Style reference image.

type StyleReferenceImage

type StyleReferenceImage struct {
	// The reference image for the editing operation.
	ReferenceImage *Image `json:"referenceImage,omitempty"`
	// The ID of the reference image.
	ReferenceID int32 `json:"referenceId,omitempty"`

	// Configuration for the style reference image.
	Config *StyleReferenceConfig `json:"config,omitempty"`
	// contains filtered or unexported fields
}

A style reference image. This encapsulates a style reference image provided by the user, and additionally optional config parameters for the style reference image. A raw reference image can also be provided as a destination for the style to be applied to.

func NewStyleReferenceImage added in v0.5.0

func NewStyleReferenceImage(referenceImage *Image, referenceID int32, config *StyleReferenceConfig) *StyleReferenceImage

NewStyleReferenceImage creates a new ControlReferenceImage.

type SubjectReferenceConfig

type SubjectReferenceConfig struct {
	// The subject type of a subject reference image.
	SubjectType SubjectReferenceType `json:"subjectType,omitempty"`
	// Subject description for the image.
	SubjectDescription string `json:"subjectDescription,omitempty"`
}

Configuration for a Subject reference image.

type SubjectReferenceImage

type SubjectReferenceImage struct {
	// The reference image for the editing operation.
	ReferenceImage *Image `json:"referenceImage,omitempty"`
	// The ID of the reference image.
	ReferenceID int32 `json:"referenceId,omitempty"`

	// Configuration for the subject reference image.
	Config *SubjectReferenceConfig `json:"config,omitempty"`
	// contains filtered or unexported fields
}

A subject reference image. This encapsulates a subject reference image provided by the user, and additionally optional config parameters for the subject reference image. A raw reference image can also be provided as a destination for the subject to be applied to.

func NewSubjectReferenceImage added in v0.5.0

func NewSubjectReferenceImage(referenceImage *Image, referenceID int32, config *SubjectReferenceConfig) *SubjectReferenceImage

NewSubjectReferenceImage creates a new SubjectReferenceImage.

type SubjectReferenceType

type SubjectReferenceType string

Enum representing the subject type of a subject reference image.

const (
	SubjectReferenceTypeSubjectTypeDefault SubjectReferenceType = "SUBJECT_TYPE_DEFAULT"
	SubjectReferenceTypeSubjectTypePerson  SubjectReferenceType = "SUBJECT_TYPE_PERSON"
	SubjectReferenceTypeSubjectTypeAnimal  SubjectReferenceType = "SUBJECT_TYPE_ANIMAL"
	SubjectReferenceTypeSubjectTypeProduct SubjectReferenceType = "SUBJECT_TYPE_PRODUCT"
)

type ThinkingConfig added in v0.1.0

type ThinkingConfig struct {
	// Indicates whether to include thoughts in the response. If true, thoughts are returned
	// only if the model supports thought and thoughts are available.
	IncludeThoughts bool `json:"includeThoughts,omitempty"`
	// Indicates the thinking budget in tokens.
	ThinkingBudget *int32 `json:"thinkingBudget,omitempty"`
}

The thinking features configuration.

type TokensInfo added in v0.1.0

type TokensInfo struct {
	// Optional. Optional fields for the role from the corresponding Content.
	Role string `json:"role,omitempty"`
	// A list of token IDs from the input.
	TokenIDs []int64 `json:"tokenIds,omitempty"`
	// A list of tokens from the input.
	Tokens [][]byte `json:"tokens,omitempty"`
}

Tokens info with a list of tokens and the corresponding list of token ids.

func (*TokensInfo) MarshalJSON added in v0.6.0

func (ti *TokensInfo) MarshalJSON() ([]byte, error)

func (*TokensInfo) UnmarshalJSON added in v0.1.0

func (ti *TokensInfo) UnmarshalJSON(data []byte) error

type Tool

type Tool struct {
	// List of function declarations that the tool supports.
	FunctionDeclarations []*FunctionDeclaration `json:"functionDeclarations,omitempty"`
	// Optional. Retrieval tool type. System will always execute the provided retrieval
	// tool(s) to get external knowledge to answer the prompt. Retrieval results are presented
	// to the model for generation.
	Retrieval *Retrieval `json:"retrieval,omitempty"`
	// Optional. Google Search tool type. Specialized retrieval tool
	// that is powered by Google Search.
	GoogleSearch *GoogleSearch `json:"googleSearch,omitempty"`
	// Optional. GoogleSearchRetrieval tool type. Specialized retrieval tool that is powered
	// by Google search.
	GoogleSearchRetrieval *GoogleSearchRetrieval `json:"googleSearchRetrieval,omitempty"`
	// Optional. CodeExecution tool type. Enables the model to execute code as part of generation.
	// This field is only used by the Gemini Developer API services.
	CodeExecution *ToolCodeExecution `json:"codeExecution,omitempty"`
}

Tool details of a tool that the model may use to generate a response.

type ToolCodeExecution

type ToolCodeExecution struct {
}

Tool that executes code generated by the model, and automatically returns the result to the model. See also [ExecutableCode]and CodeExecutionResult which are input and output to this tool.

type ToolConfig

type ToolConfig struct {
	// Optional. Function calling config.
	FunctionCallingConfig *FunctionCallingConfig `json:"functionCallingConfig,omitempty"`
}

Tool config. This config is shared for all tools provided in the request.

type TrafficType added in v1.0.0

type TrafficType string

Traffic type. This shows whether a request consumes Pay-As-You-Go or Provisioned Throughput quota.

const (
	// Unspecified request traffic type.
	TrafficTypeUnspecified TrafficType = "TRAFFIC_TYPE_UNSPECIFIED"
	// Type for Pay-As-You-Go traffic.
	TrafficTypeOnDemand TrafficType = "ON_DEMAND"
	// Type for Provisioned Throughput traffic.
	TrafficTypeProvisionedThroughput TrafficType = "PROVISIONED_THROUGHPUT"
)

type Transcription added in v1.1.0

type Transcription struct {
	// Transcription text.
	Text string `json:"text,omitempty"`
	// The bool indicates the end of the transcription.
	Finished bool `json:"finished,omitempty"`
}

Audio transcription in Server Conent.

type TunedModelInfo added in v0.4.0

type TunedModelInfo struct {
	// ID of the base model that you want to tune.
	BaseModel string `json:"baseModel,omitempty"`
	// Date and time when the base model was created.
	CreateTime time.Time `json:"createTime,omitempty"`
	// Date and time when the base model was last updated.
	UpdateTime time.Time `json:"updateTime,omitempty"`
}

A tuned machine learning model.

func (*TunedModelInfo) MarshalJSON added in v0.6.0

func (c *TunedModelInfo) MarshalJSON() ([]byte, error)

type TurnCoverage added in v1.0.0

type TurnCoverage string

Options about which input is included in the user's turn.

const (
	// If unspecified, the default behavior is `TURN_INCLUDES_ONLY_ACTIVITY`.
	TurnCoverageUnspecified TurnCoverage = "TURN_COVERAGE_UNSPECIFIED"
	// The users turn only includes activity since the last turn, excluding inactivity (e.g.
	// silence on the audio stream). This is the default behavior.
	TurnCoverageTurnIncludesOnlyActivity TurnCoverage = "TURN_INCLUDES_ONLY_ACTIVITY"
	// The users turn includes all realtime input since the last turn, including inactivity
	// (e.g. silence on the audio stream).
	TurnCoverageTurnIncludesAllInput TurnCoverage = "TURN_INCLUDES_ALL_INPUT"
)

type Type

type Type string

The type of the data.

const (
	// Not specified, should not be used.
	TypeUnspecified Type = "TYPE_UNSPECIFIED"
	// OpenAPI string type
	TypeString Type = "STRING"
	// OpenAPI number type
	TypeNumber Type = "NUMBER"
	// OpenAPI integer type
	TypeInteger Type = "INTEGER"
	// OpenAPI boolean type
	TypeBoolean Type = "BOOLEAN"
	// OpenAPI array type
	TypeArray Type = "ARRAY"
	// OpenAPI object type
	TypeObject Type = "OBJECT"
)

type UpdateCachedContentConfig added in v0.1.0

type UpdateCachedContentConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// The TTL for this resource. The expiration time is computed: now + TTL.
	TTL time.Duration `json:"ttl,omitempty"`
	// Timestamp of when this resource is considered expired.
	ExpireTime time.Time `json:"expireTime,omitempty"`
}

Optional parameters for caches.update method.

func (*UpdateCachedContentConfig) MarshalJSON added in v0.6.0

func (c *UpdateCachedContentConfig) MarshalJSON() ([]byte, error)

func (*UpdateCachedContentConfig) UnmarshalJSON added in v0.7.0

func (c *UpdateCachedContentConfig) UnmarshalJSON(data []byte) error

type UpdateModelConfig added in v0.4.0

type UpdateModelConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`

	DisplayName string `json:"displayName,omitempty"`

	Description string `json:"description,omitempty"`
}

type UploadFileConfig

type UploadFileConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// The name of the file in the destination (e.g., 'files/sample-image'. If not provided
	// one will be generated.
	Name string `json:"name,omitempty"`
	// mime_type: The MIME type of the file. If not provided, it will be inferred from the
	// file extension.
	MIMEType string `json:"mimeType,omitempty"`
	// Optional display name of the file.
	DisplayName string `json:"displayName,omitempty"`
}

Used to override the default configuration.

type UpscaleImageConfig

type UpscaleImageConfig struct {
	// Used to override HTTP request options.
	HTTPOptions *HTTPOptions `json:"httpOptions,omitempty"`
	// Whether to include a reason for filtered-out images in the
	// response.
	IncludeRAIReason bool `json:"includeRaiReason,omitempty"`
	// The image format that the output should be saved as.
	OutputMIMEType string `json:"outputMimeType,omitempty"`
	// The level of compression if the OutputMIMEType is image/jpeg.
	OutputCompressionQuality *int32 `json:"outputCompressionQuality,omitempty"`
}

Configuration for upscaling an image. For more information on this configuration, refer to the `Imagen API reference documentation <https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/imagen-api>`_.

type UpscaleImageResponse added in v0.4.0

type UpscaleImageResponse struct {
	// Generated images.
	GeneratedImages []*GeneratedImage `json:"generatedImages,omitempty"`
}

type UsageMetadata added in v1.0.0

type UsageMetadata struct {
	// Number of tokens in the prompt. When `cached_content` is set, this is still the total
	// effective prompt size meaning this includes the number of tokens in the cached content.
	PromptTokenCount int32 `json:"promptTokenCount,omitempty"`
	// Number of tokens in the cached part of the prompt (the cached content).
	CachedContentTokenCount int32 `json:"cachedContentTokenCount,omitempty"`
	// Total number of tokens across all the generated response candidates.
	ResponseTokenCount int32 `json:"responseTokenCount,omitempty"`
	// Number of tokens present in tool-use prompt(s).
	ToolUsePromptTokenCount int32 `json:"toolUsePromptTokenCount,omitempty"`
	// Number of tokens of thoughts for thinking models.
	ThoughtsTokenCount int32 `json:"thoughtsTokenCount,omitempty"`
	// Total token count for prompt, response candidates, and tool-use prompts(if present).
	TotalTokenCount int32 `json:"totalTokenCount,omitempty"`
	// List of modalities that were processed in the request input.
	PromptTokensDetails []*ModalityTokenCount `json:"promptTokensDetails,omitempty"`
	// List of modalities that were processed in the cache input.
	CacheTokensDetails []*ModalityTokenCount `json:"cacheTokensDetails,omitempty"`
	// List of modalities that were returned in the response.
	ResponseTokensDetails []*ModalityTokenCount `json:"responseTokensDetails,omitempty"`
	// List of modalities that were processed in the tool-use prompt.
	ToolUsePromptTokensDetails []*ModalityTokenCount `json:"toolUsePromptTokensDetails,omitempty"`
	// Traffic type. This shows whether a request consumes Pay-As-You-Go
	// or Provisioned Throughput quota.
	TrafficType TrafficType `json:"trafficType,omitempty"`
}

Usage metadata about response(s).

type VertexAISearch

type VertexAISearch struct {
	// Optional. Fully-qualified Vertex AI Search data store resource ID. Format: `projects/{project}/locations/{location}/collections/{collection}/dataStores/{dataStore}`
	Datastore string `json:"datastore,omitempty"`
	// Optional. Fully-qualified Vertex AI Search engine resource ID. Format: `projects/{project}/locations/{location}/collections/{collection}/engines/{engine}`
	Engine string `json:"engine,omitempty"`
}

Retrieve from Vertex AI Search datastore or engine for grounding. datastore and engine are mutually exclusive. See https://cloud.google.com/products/agent-builder

type VertexRAGStore

type VertexRAGStore struct {
	// Optional. Deprecated. Please use rag_resources instead.
	RAGCorpora []string `json:"ragCorpora,omitempty"`
	// Optional. The representation of the RAG source. It can be used to specify corpus
	// only or ragfiles. Currently only support one corpus or multiple files from one corpus.
	// In the future we may open up multiple corpora support.
	RAGResources []*VertexRAGStoreRAGResource `json:"ragResources,omitempty"`
	// Optional. The retrieval config for the RAG query.
	RAGRetrievalConfig *RAGRetrievalConfig `json:"ragRetrievalConfig,omitempty"`
	// Optional. Number of top k results to return from the selected corpora.
	SimilarityTopK *int32 `json:"similarityTopK,omitempty"`
	// Optional. Only return results with vector distance smaller than the threshold.
	VectorDistanceThreshold *float64 `json:"vectorDistanceThreshold,omitempty"`
}

Retrieve from Vertex RAG Store for grounding. You can find API default values and more details at https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/rag-api-v1#parameters-list

type VertexRAGStoreRAGResource

type VertexRAGStoreRAGResource struct {
	// Optional. RAGCorpora resource name. Format: `projects/{project}/locations/{location}/ragCorpora/{rag_corpus}`
	RAGCorpus string `json:"ragCorpus,omitempty"`
	// Optional. rag_file_id. The files should be in the same rag_corpus set in rag_corpus
	// field.
	RAGFileIDs []string `json:"ragFileIds,omitempty"`
}

The definition of the RAG resource.

type Video added in v0.7.0

type Video struct {
	// Path to another storage.
	URI string `json:"uri,omitempty"`
	// Video bytes.
	VideoBytes []byte `json:"videoBytes,omitempty"`
	// Video encoding, for example "video/mp4".
	MIMEType string `json:"mimeType,omitempty"`
}

A generated video.

type VideoMetadata

type VideoMetadata struct {
	// Optional. The end offset of the video.
	EndOffset time.Duration `json:"endOffset,omitempty"`
	// Optional. The start offset of the video.
	StartOffset time.Duration `json:"startOffset,omitempty"`
}

Metadata describes the input video content.

func (*VideoMetadata) MarshalJSON added in v0.6.0

func (c *VideoMetadata) MarshalJSON() ([]byte, error)

func (*VideoMetadata) UnmarshalJSON added in v0.6.0

func (c *VideoMetadata) UnmarshalJSON(data []byte) error

type VoiceConfig

type VoiceConfig struct {
	// The configuration for the speaker to use.
	PrebuiltVoiceConfig *PrebuiltVoiceConfig `json:"prebuiltVoiceConfig,omitempty"`
}

The configuration for the voice to use.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL