ollamarunner

package

v0.0.0-...-6e6905b Latest Latest Go to latest Published: Sep 22, 2025 License: MIT Imports: 36 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/zhuangjie1125/ollama

Links

Open Source Insights

Documentation ¶

Index ¶

func Execute(args []string) error
type ErrReprocessInputs
- func (e *ErrReprocessInputs) Error() string
type InputCache
- func NewInputCache(model model.Model, kvCacheType string, kvSize int32, numSlots int, ...) (*InputCache, error)
type InputCacheSlot
type NewSequenceParams
type Sequence
type Server
- func (s *Server) NewSequence(prompt string, images []llm.ImageData, params NewSequenceParams) (*Sequence, error)

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func Execute ¶

func Execute(args []string) error

Types ¶

type ErrReprocessInputs ¶

type ErrReprocessInputs struct {
	Inputs []*input.Input
}

func (*ErrReprocessInputs) Error ¶

func (e *ErrReprocessInputs) Error() string

type InputCache ¶

type InputCache struct {
	// contains filtered or unexported fields
}

func NewInputCache ¶

func NewInputCache(model model.Model, kvCacheType string, kvSize int32, numSlots int, batchSize int, multiUserCache bool) (*InputCache, error)

func (*InputCache) Close ¶

func (c *InputCache) Close()

func (*InputCache) LoadCacheSlot ¶

func (c *InputCache) LoadCacheSlot(prompt []*input.Input, cachePrompt bool) (*InputCacheSlot, []*input.Input, error)

func (*InputCache) ShiftCacheSlot ¶

func (c *InputCache) ShiftCacheSlot(slot *InputCacheSlot, numKeep int32) error

Frees up space in the KV cache by deleting the oldest half of history and shifting the newest half into that space (saving numKeep inputs at the beginning).

Assumes that at least 1 entry can be freed up by shifting (i.e. numKeep < numCtx)

func (*InputCache) ShiftDiscard ¶

func (c *InputCache) ShiftDiscard(inputLen int32, numKeep int32) int32

TODO(jessegross): If we need to reprocess the inputs we should ensure that we don't split up a SameBatch

type InputCacheSlot ¶

type InputCacheSlot struct {
	// Index in the KV cache
	Id int

	// Inputs that are stored in the KV cache
	Inputs []*input.Input

	// is this cache actively being processed as part of a sequence?
	InUse bool
	// contains filtered or unexported fields
}

type NewSequenceParams ¶

type NewSequenceParams struct {
	// contains filtered or unexported fields
}

type Sequence ¶

type Sequence struct {
	// contains filtered or unexported fields
}

type Server ¶

type Server struct {
	// contains filtered or unexported fields
}

func (*Server) NewSequence ¶

func (s *Server) NewSequence(prompt string, images []llm.ImageData, params NewSequenceParams) (*Sequence, error)

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL