ratelimit

package

v0.0.0-...-1ce1ff6 Latest Latest Go to latest Published: Aug 5, 2025 License: Apache-2.0 Imports: 18 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/wso2/apk

Links

Open Source Insights

Documentation ¶

Index ¶

Constants
func ReadGzip(gzipData []byte) (string, error)
type AIRatelimitHelper
- func NewAIRatelimitHelper(cfg *config.Server) *AIRatelimitHelper
- func (airl *AIRatelimitHelper) DoAIRatelimit(tokenCount TokenCountAndModel, doBackendBasedAIRatelimit bool, ...)
type TokenCountAndModel
- func ExtractTokenCountFromExternalProcessingResponseBody(body []byte, promptPath, completionPath, totalPath, modelPath string) (*TokenCountAndModel, error)
- func ExtractTokenCountFromExternalProcessingResponseHeaders(headerValues []*v3.HeaderValue, ...) (*TokenCountAndModel, error)

Constants ¶

View Source

const (
	// DescriptorKeyForAIPromtTokenCount is the descriptor key for the AI prompt token count.
	DescriptorKeyForAIPromtTokenCount = "airequesttokencount"
	// DescriptorKeyForAICompletionTokenCount is the descriptor key for the AI completion token count.
	DescriptorKeyForAICompletionTokenCount = "airesponsetokencount"
	// DescriptorKeyForAITotalTokenCount is the descriptor key for the AI total token count.
	DescriptorKeyForAITotalTokenCount = "aitotaltokencount"
	// DescriptorKeyForSubscriptionBasedAIRequestTokenCount is the descriptor key for the subscription-based AI request token count.
	DescriptorKeyForSubscriptionBasedAIRequestTokenCount = "airequesttokencountsubs"
	// DescriptorKeyForSubscriptionBasedAIResponseTokenCount is the descriptor key for the subscription-based AI response token count.
	DescriptorKeyForSubscriptionBasedAIResponseTokenCount = "airesponsetokencountsubs"
	// DescriptorKeyForSubscriptionBasedAITotalTokenCount is the descriptor key for the subscription-based AI total token count.
	DescriptorKeyForSubscriptionBasedAITotalTokenCount = "aitotaltokencountsubs"
	// DescriptorKeyForAISubscription is the descriptor key for the AI subscription.
	DescriptorKeyForAISubscription = "subscription"
)

Variables ¶

This section is empty.

Functions ¶

func ReadGzip ¶

func ReadGzip(gzipData []byte) (string, error)

ReadGzip decompresses a GZIP-compressed byte slice and returns the string output

Types ¶

type AIRatelimitHelper ¶

type AIRatelimitHelper struct {
	// contains filtered or unexported fields
}

AIRatelimitHelper is a helper struct for managing AI rate limiting.

func NewAIRatelimitHelper ¶

func NewAIRatelimitHelper(cfg *config.Server) *AIRatelimitHelper

NewAIRatelimitHelper creates a new instance of the AIRatelimitHelper.

func (*AIRatelimitHelper) DoAIRatelimit ¶

func (airl *AIRatelimitHelper) DoAIRatelimit(tokenCount TokenCountAndModel, doBackendBasedAIRatelimit bool, doSubscriptionBasedAIRatelimit bool, backendBasedAIRatelimitDescriptorValue string, subscription *subscription_model.Subscription, application *subscription_model.Application)

DoAIRatelimit performs AI rate limiting.

type TokenCountAndModel ¶

type TokenCountAndModel struct {
	Prompt     int
	Completion int
	Total      int
	Model      string
}

TokenCountAndModel is a struct that holds the prompt, completion, and total token counts.

func ExtractTokenCountFromExternalProcessingResponseBody ¶

func ExtractTokenCountFromExternalProcessingResponseBody(body []byte, promptPath, completionPath, totalPath, modelPath string) (*TokenCountAndModel, error)

ExtractTokenCountFromExternalProcessingResponseBody extracts token counts from external processing response body.

func ExtractTokenCountFromExternalProcessingResponseHeaders ¶

func ExtractTokenCountFromExternalProcessingResponseHeaders(headerValues []*v3.HeaderValue, promptHeader, completionHeader, totalHeader, modelHeader string) (*TokenCountAndModel, error)

ExtractTokenCountFromExternalProcessingResponseHeaders extracts token counts from external processing response headers.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL