Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Router ¶
type Router struct {
// contains filtered or unexported fields
}
func NewRouter ¶
func NewRouter(serverConfigs []server.ServerConfig, strategyType RouterStrategyType) (*Router, error)
NewRouter creates a new Router instance with the given server configurations and strategy type. It returns a pointer to the Router and an error if any. The serverConfigs parameter is a slice of server.ServerConfig that contains the configurations for each server. The strategyType parameter is the type of router strategy to be used. If the serverConfigs slice is empty, it returns an error with the message "empty server config". Otherwise, it creates a new RouterServer for each server configuration and adds them to the servers slice. Finally, it initializes the Router with the servers, serverCount, requestCount, and strategy.
func (*Router) GetChatCompletions ¶
func (r *Router) GetChatCompletions(ctx context.Context, body openai.ChatCompletionNewParams, opts ...option.RequestOption) (*openai.ChatCompletion, error)
GetChatCompletions - Gets chat completions for the provided chat messages. Completions support a wide variety of tasks and generate text that continues from or "completes" provided prompt data. If the operation fails it returns an *azcore.ResponseError type.
func (*Router) GetChatCompletionsStream ¶
func (r *Router) GetChatCompletionsStream(ctx context.Context, body openai.ChatCompletionNewParams, opts ...option.RequestOption) (*ssestream.Stream[openai.ChatCompletionChunk], error)
GetChatCompletionsStream - Return the chat completions for a given prompt as a sequence of events. If the operation fails it returns an *azcore.ResponseError type.
- options - GetCompletionsOptions contains the optional parameters for the Client.GetCompletions method.
type RouterStrategyType ¶
type RouterStrategyType string
const ( //Simple Round Robin Strategy to get a server RoundRobinStrategy RouterStrategyType = "round-robin" //Least Connection Strategy to get a server using active connections LeastConnectionStrategy RouterStrategyType = "least-connection" //Least Average Strategy to get a server LeastLatencyStrategy RouterStrategyType = "least-latency" )
Source Files