Documentation
¶
Index ¶
- Constants
- type Bounded
- type BytesCollector
- type BytesIngest
- type BytesSource
- type Closable
- type Connector
- type EOFIngest
- type ErrorIngest
- type Function
- type FunctionContext
- type HasDynamicProps
- type Logger
- type LookupBytesSource
- type LookupSource
- type MessageTuple
- type MessageTupleList
- type MetaInfo
- type ModuleInfo
- type Nodelet
- type PullBytesSource
- type PullTupleSource
- type RawTuple
- type ReadonlyMessage
- type Rewindable
- type Sink
- type Source
- type StatusChangeHandler
- type Store
- type StreamContext
- type TupleCollector
- type TupleIngest
- type TupleSource
Constants ¶
const ( ConnectionConnected = "connected" ConnectionConnecting = "connecting" ConnectionDisconnected = "disconnected" )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Bounded ¶
type Bounded interface {
SetEofIngest(eof EOFIngest)
}
Bounded means the source can have an end.
type BytesCollector ¶
type BytesCollector interface { Sink Collect(ctx StreamContext, item RawTuple) error }
type BytesIngest ¶
type BytesSource ¶
type BytesSource interface { Source Subscribe(ctx StreamContext, ingest BytesIngest, ingestError ErrorIngest) error }
BytesSource receives the bytes payload pushed by the external source
type Closable ¶
type Closable interface {
Close(ctx StreamContext) error
}
type Connector ¶
type Connector interface {
Connect(ctx StreamContext, sch StatusChangeHandler) error
}
Connector is a source feature that allows the source to connect to the data source.
type EOFIngest ¶
type EOFIngest func(ctx StreamContext, msg string)
type ErrorIngest ¶
type ErrorIngest func(ctx StreamContext, err error)
type Function ¶
type Function interface { // Validate The argument is a list of xsql.Expr Validate(args []any) error // Exec Execute the function, return the result and if execution is successful. // If execution fails, return the error and false. Exec(ctx FunctionContext, args []any) (interface{}, bool) // IsAggregate If this function is an aggregate function. Each parameter of an aggregate function will be a slice IsAggregate() bool }
type FunctionContext ¶
type FunctionContext interface { StreamContext GetFuncId() int }
type HasDynamicProps ¶
type Logger ¶
type Logger interface { Debug(args ...interface{}) Info(args ...interface{}) Warn(args ...interface{}) Error(args ...interface{}) Debugf(format string, args ...interface{}) Infof(format string, args ...interface{}) Warnf(format string, args ...interface{}) Errorf(format string, args ...interface{}) }
type LookupBytesSource ¶
type LookupBytesSource interface { Source // Lookup receive multiple rows of bytes Lookup(ctx StreamContext, fields []string, keys []string, values []any) ([][]byte, error) }
LookupBytesSource looks up with the bytes payload pushed by the external source
type LookupSource ¶
type LookupSource interface { Source // Lookup receive lookup values to construct the query and return query results Lookup(ctx StreamContext, fields []string, keys []string, values []any) ([]map[string]any, error) }
LookupSource is a source feature to query the source on demand
type MessageTuple ¶
type MessageTuple interface { ReadonlyMessage }
MessageTuple is an interface of the below interfaces
type MessageTupleList ¶
type ModuleInfo ¶
type Nodelet ¶
type Nodelet interface { // Provision is called when the node is created, usually setting the configs. Do not put time-consuming operations here. Provision(ctx StreamContext, configs map[string]any) error Closable }
type PullBytesSource ¶
type PullBytesSource interface { Source Pull(ctx StreamContext, trigger time.Time, ingest BytesIngest, ingestError ErrorIngest) }
PullBytesSource fetch the bytes payload in an interval from the external source. Interval property must be defined
type PullTupleSource ¶
type PullTupleSource interface { Source Pull(ctx StreamContext, trigger time.Time, ingest TupleIngest, ingestError ErrorIngest) }
PullTupleSource fetch the non-bytes payload in an interval from the external source. Interval property must be defined
type ReadonlyMessage ¶
ReadonlyMessage Message is the interface that wraps each record. Use this interface to exchange data between different components. It is used in sink
type Rewindable ¶
type Rewindable interface { GetOffset() (any, error) Rewind(offset any) error ResetOffset(input map[string]any) error }
Rewindable is a source feature that allows the source to rewind to a specific offset.
type Sink ¶
Sink is the interface that wraps the basic Sink method. It is used to connect to the external system and send data to it. A sink must implement the Sink interface AND any collector interface. The lifecycle of a sink: Provision -> Connect -> Collect -> Close
type Source ¶
Source is the raw interface that wraps the basic Source method. It cannot be used independently, must implement more traits. The lifecycle of a source: Provision -> Connect -> Subscribe/Pull -> Close
type StatusChangeHandler ¶
type StreamContext ¶
type StreamContext interface { context.Context GetLogger() Logger GetRuleId() string GetOpId() string GetInstanceId() int GetRootPath() string WithMeta(ruleId string, opId string, store Store) StreamContext WithInstance(instanceId int) StreamContext WithCancel() (StreamContext, context.CancelFunc) EnableTracer(enabled bool) IsTraceEnabled() bool SetError(e error) // IncrCounter State handling IncrCounter(key string, amount int) error GetCounter(key string) (int, error) PutState(key string, value interface{}) error GetState(key string) (interface{}, error) DeleteState(key string) error // ParseTemplate parse the template string with the given data ParseTemplate(template string, data interface{}) (string, error) // ParseJsonPath parse the jsonPath string with the given data ParseJsonPath(jsonPath string, data interface{}) (interface{}, error) }
type TupleCollector ¶
type TupleCollector interface { Sink Collect(ctx StreamContext, item MessageTuple) error CollectList(ctx StreamContext, items MessageTupleList) error }
type TupleIngest ¶
TupleIngest reads in a structural data or its list. It supports map and []map for now
type TupleSource ¶
type TupleSource interface { Source Subscribe(ctx StreamContext, ingest TupleIngest, ingestError ErrorIngest) error }
TupleSource receives the non-bytes payload pushed by the external source