flume

package module
v2.0.0-rc4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 21, 2026 License: MIT Imports: 16 Imported by: 0

README

Flume GoDoc Go Report Card Build

Flume is a handler for the log/slog package.

Usage

go get github.com/ThalesGroup/flume/v2

Example:

import github.com/ThalesGroup/flume/v2

// configure logging from the environment, e.g.:
// FLUME={"level":"INFO", "levels":{"http":"DEBUG"}, "handler":"json"}
flume.MustConfigFromEnv()

l := flume.New("http")  // equivalent to slog.New(flume.Default()).With(flume.LoggerKey, "http")
l.Debug("hello world")

Features

A flume handler has a couple nifty capabilities:

  • It enables different levels based on the value of a special logger attribute in the record. For example, the default level might be INFO, but records with the attribute logger=http can be enabled at the debug level. (The attribute name is configurable).
  • Flume handlers forward records to another, "sink" Handler. The sink can be changed at runtime, in an atomic, concurrency-safe way, even after loggers are created and while they are being used.
    So you can switch from text to json at runtime without re-creating loggers, enable/disable the source attribute, or add/remove ReplaceAttr functions.
  • Middleware: you can configure Handler middleware, which can do things like augmenting log records with additional attributes from the context. As with the sink handler, middleware can be added or swapped at runtime.
  • Flume's HandlerOptions can be configured from a JSON configuration spec, and has convenience methods for reading this configuration from environment variables.
  • Integrates with github.com/ansel1/console-slog, providing a very human-friendly output format

Migration from v1

Flume v1 was based on zap internally, but its logging functions (e.g. log.Info()) had the same signatures as slog.Logger.

Migration steps:

  • Replace flume imports with flume/v2
  • Replace flume.New( with flume.New( (or with calls to slog.New() passing a flume/v2 handler)
  • Replace .IsDebug() with .Enabled(ctx, slog.LevelDebug)
  • Compiler errors: fix unmatched arguments to logging function (e.g. l.Info("temp", temp) -> l.Info("temp", "value", temp)
  • Consider zap2slog to do the transition incrementally

Contributing

To build, be sure to have a recent go SDK, golangci-lint, and just. Then run just.

Merge requests are welcome!

Documentation

Index

Examples

Constants

View Source
const (
	TextHandler      = "text"
	JSONHandler      = "json"
	TermHandler      = "term"
	TermColorHandler = "term-color"
	NoopHandler      = "noop"
)
View Source
const (
	LevelDebug = slog.LevelDebug
	LevelInfo  = slog.LevelInfo
	LevelWarn  = slog.LevelWarn
	LevelError = slog.LevelError
	LevelOff   = slog.Level(math.MaxInt)
	LevelAll   = slog.Level(math.MinInt)
)
View Source
const (
	// LoggerKey is the key which stores the name of the logger.  The name was the argument
	// passed to Controller.NewLogger() or Controller.NewHandler()
	LoggerKey = "logger"
)

Variables

View Source
var (
	ErrInvalidLevels       = errors.New("invalid levels value")
	ErrInvalidLevel        = errors.New("invalid log level")
	ErrUnregisteredHandler = errors.New("unregistered handler")
)

Define static error variables

View Source
var DefaultConfigEnvVars = []string{"FLUME"}

DefaultConfigEnvVars is a list of the environment variables that ConfigFromEnv will search by default.

Functions

func AbbreviateLevel

func AbbreviateLevel(_ []string, attr slog.Attr) slog.Attr

AbbreviateLevel is a ReplaceAttr function that abbreviates log level names.

It modifies the attribute if it's a log level (slog.Level) and changes the level name to its abbreviation. The abbreviations are:

  • "DEBUG" becomes "DBG"
  • "INFO" becomes "INF"
  • "WARN" becomes "WRN"
  • "ERROR" becomes "ERR"

If the attribute's value is not a slog.Level, it is returned unchanged.

Example:

// Create a logger with the ReplaceAttr function.
logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{
	ReplaceAttr: AbbreviateLevel,
}))

// Log a message.
logger.Debug("This is a debug message.") // Output will be: level=DBG msg="This is a debug message."

func ChainReplaceAttrs

func ChainReplaceAttrs(fns ...func([]string, slog.Attr) slog.Attr) func([]string, slog.Attr) slog.Attr

ChainReplaceAttrs composes a series of ReplaceAttr functions into a single ReplaceAttr function.

func ConfigFromEnv

func ConfigFromEnv(envvars ...string) error

ConfigFromEnv configures flume from environment variables. It should be called from main():

func main() {
    flume.ConfigFromEnv()
    ...
 }

It searches envvars for the first environment variable that is set, and attempts to parse the value.

If no environment variable is set, it silently does nothing.

If an environment variable with a value is found, but parsing fails, an error is returned.

If envvars is empty, it defaults to DefaultConfigEnvVars.

func DetailedErrors

func DetailedErrors(_ []string, a slog.Attr) slog.Attr

DetailedErrors is a ReplaceAttr function which ensures that errors implement fmt.Formatter are rendered the same by the JSONHandler as by the TextHandler. By default, the slog.JSONHandler renders errors as err.Error(), where as the slog.TextHandler will render it with fmt.Sprintf("%+v", err).

When using DetailedErrors, if any error values implement fmt.Formatter ( and therefore *may* implement some form of detailed error printing), and *do not* already implement json.Marshaler, then the value is wrapped in an error implementation which implements json.Marshaler, and marshals the error to a string using fmt.Sprintf("%+v", err).

func FixedTime

func FixedTime(t time.Time) func(_ []string, a slog.Attr) slog.Attr

FixedTime replaces all time attributes with a fixed value. Useful in testing to get a predictable log line.

func FormatTimes

func FormatTimes(format string) func([]string, slog.Attr) slog.Attr

FormatTimes returns a ReplaceAttr function that formats time values according to the specified format.

It modifies an attribute if its value is a time.Time. The time is formatted using the provided format string, and the attribute's value is updated to the formatted time string.

If the attribute's value is not a time.Time, it is returned unchanged.

Args:

format string: The format string to use for formatting time values (e.g., time.DateTime, "2006-01-02", "15:04:05").

Returns:

func([]string, slog.Attr) slog.Attr: A ReplaceAttr function that formats time values.

Example:

	// Create a ReplaceAttr function that formats times using a custom format.
	customTimeFormat := FormatTimes("2006-01-02 15:04:05")

	// Create a logger with the ReplaceAttr function.
	logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{
	    ReplaceAttr: customTimeFormat,
	}))

	// Log a message with a time attribute.
	logger.Info("Time example", slog.Time("now", time.Now()))
	// Output might be: level=INFO msg="Time example" now="2023-10-27 10:30:00"

 // Create a ReplaceAttr function that formats times using a predefined format.
 dateTimeFormat := FormatTimes(time.DateTime)

 // Create a logger with the ReplaceAttr function.
 logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{
     ReplaceAttr: dateTimeFormat,
 }))

 // Log a message with a time attribute.
 logger.Info("Time example", slog.Time("now", time.Now()))
 // Output might be: level=INFO msg="Time example" now="2023-10-27 10:30:00.000"

func ISO8601Time

func ISO8601Time() func([]string, slog.Attr) slog.Attr

ISO8601Time returns a ReplaceAttr function that formats time to the ISO8601 compliant format used by flumev1 and zap. It is here to provide backward compatibility with flume v1, but should probably be avoided for new applications, in favor of the default formatting selected by the handler. The json and text handlers in the slog package use RFC3339, which is mostly compatible with ISO8601 anyway (see https://ijmacd.github.io/rfc3339-iso8601/ for how they overlap).

func LogFuncWriter

func LogFuncWriter(l func(args ...any), trimSpace bool) io.Writer

LogFuncWriter is a writer which writes to a logging function signature like that of testing.T.Log() and fmt/log.Println(). It can be used to redirect slog output (or any other line-oriented logging output) to some other logging function.

SetOut(LogFuncWriter(fmt.Println, true))
SetOut(LogFuncWriter(t.Log, true))

func LoggerFuncWriter

func LoggerFuncWriter(l func(msg string, kvpairs ...any)) io.Writer

LoggerFuncWriter is a writer which writes to functions with a signature like slog.Logger's logging functions, like slog.Logger.Info(), slog.Logger.Debug(), and slog.Logger.Error(). It can be used to adapt libraries which expect a logging function to slog.Logger.

http.Server{
    ErrorLog: log.New(LoggerFuncWriter(flume.New("http").Error), "", 0),
}

func MustConfigFromEnv

func MustConfigFromEnv(envvars ...string)

MustConfigFromEnv is like ConfigFromEnv, but panics on error.

func New

func New(name string) *slog.Logger

New is a convenience for creating a named logger using the default handler.

These package-level functions are typically used to initialize package-level loggers in var initializers, which can later be configured in main(). See [Controller.Logger]

Example:

package http
var logger = flume.New("http")
Example
ogOpts := Default().HandlerOptions()
defer Default().SetHandlerOptions(ogOpts)

// The default handler is a noop handler that discards all log messages.
// To enable logging, using a default slog text handler writing to os.Stdout,
// call Default().SetHandlerOptions(nil)
Default().SetHandlerOptions(nil)

// This creates a named logger with the name "my-app", using the default handler.
log := New("my-app")
log.Info("Hello, World!")

// Output will be something like:
// time=2025-02-26T09:47:59.129-06:00 level=INFO msg="Hello, World!" logger=my-app

func RFC3339MillisTime

func RFC3339MillisTime() func(_ []string, a slog.Attr) slog.Attr

RFC3339MillisTime returns a ReplaceAttr function which formats time to the format that slog's text handler uses.

func RegisterHandlerFn

func RegisterHandlerFn(name string, fn HandlerFn)

RegisterHandlerFn registers a handler with a name. The handler can be looked up with LookupHandlerFn. If a handler function was already registered with the given name, the old handler function is replaced. Built-in handler functions can also be replaced in this manner.

func SecondsDuration

func SecondsDuration() func(_ []string, a slog.Attr) slog.Attr

SecondsDuration formats durations as fractional seconds. This may be used to mimic the default encoding of durations in flumev1/zap.

func SimpleTime

func SimpleTime() func([]string, slog.Attr) slog.Attr

SimpleTime returns a ReplaceAttr function that formats time values to a simple time format: "15:04:05.000".

It's a convenience function that uses FormatTimes internally with a predefined format.

Example:

// Create a logger with the ReplaceAttr function.
logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{
    ReplaceAttr: SimpleTime(),
}))

// Log a message with a time attribute.
logger.Info("Time example", slog.Time("now", time.Now()))
// Output might be: level=INFO msg="Time example" now="10:30:00.000"

func UnmarshalEnv

func UnmarshalEnv(o *HandlerOptions, envvars ...string) error

UnmarshalEnv reads handler options from an environment variable. The first environment variable in the list with a non-empty value will be unmarshaled into the options arg.

The first argument must not be nil.

The value of the environment variable can be either json, or a levels string:

FLUME={"level":"inf"}            // json
FLUME=*=inf                      // levels string

This is the full schema for the json encoding:

{
  "development": <bool>,
  "handler": <str>,       // looks up HandlerFn using LookupHandlerFn()
  "encoding": <str>,      // v1 alias for "handler"; if both set, "handler" wins
  "level": <str>,         // e.g. "INF", "INFO", "INF-1"
  "levels": <str or obj>, // either a levels string, or an object where the keys
                          // are logger names, and the values are levels (in the same
                          // format as the "level" property)
  "addSource": <bool>,
  "addCaller": <bool>,    // v1 alias for "addSource"; if both set, "addSource" wins
}

Level strings are in the form:

	 Levels    = Directive {"," Directive} .
  Directive = logger | "-" logger | logger "=" Level | "*" .
  Level     = LevelName [ "-" offset ] | int .
  LevelName = "DEBUG" | "DBG" | "INFO" | "INF" |
              "WARN" | "WRN" | "ERROR" | "ERR" |
              "ALL" | "OFF" | ""

Where `logger` is the name of a logger. "*" sets the default level. LevelName is case-insensitive.

Example:

*=INF,http,-sql,boot=DEBUG,authz=ERR,authn=INF+1,keys=4

- sets default level to info - enables all log levels on the http logger - disables all logging from the sql logger - sets the boot logger to debug - sets the authz logger to ERR - sets the authn logger to level 1 (slog.LevelInfo + 1) - sets the keys logger to WARN (slog.LevelWarn == 4)

func V1JSONHandler

func V1JSONHandler()

V1JSONHandler configures the default json to behave like flume v1: - abbreviations for levels - durations in seconds - "errorsVerbose" key for formattable errors

func V1VerboseErrors

func V1VerboseErrors(_ []string, a slog.Attr) slog.Attr

V1VerboseErrors is a ReplaceAttr function which is meant to mimic how errors were rendered in flumev1. In flumev1, an error which also implemented fmt.Formatter would result in two attributes being added to the log message: "error" and "errorVerbose". "error" would only contain the error's message, and "errorVerbose" would contain the error object formatted with fmt.Sprintf(), which typically includes the error's stacktrace or other metadata.

flumev2 has a DetailedErrors function, but that just replaces the "error" value in place with the formatted error value. We need something different to mimic flumev1/zap. See zap.NamedError() for more details.

Types

type FlumeV1Logger

type FlumeV1Logger interface {
	Debug(msg string, args ...any)
	Info(msg string, args ...any)
	Error(msg string, args ...any)

	IsDebug() bool
	IsInfo() bool
}

FlumeV1Logger describes the most commonly used parts of the flumev1 Logger API. In most cases, references to flume/yugolog.Logger could be replaced with this interface without breaking callers.

type Handler

type Handler struct {
	// contains filtered or unexported fields
}

func Default

func Default() *Handler

Default returns the default Handler. It will never be nil. By default, it is a noop handler that discards all log messages.

The simplest way to enable logging is to call Default().SetHandlerOptions(nil)

func NewHandler

func NewHandler(w io.Writer, opts *HandlerOptions) *Handler

func (*Handler) Enabled

func (h *Handler) Enabled(ctx context.Context, lvl slog.Level) bool

func (*Handler) Handle

func (h *Handler) Handle(ctx context.Context, rec slog.Record) error

func (*Handler) HandlerOptions

func (h *Handler) HandlerOptions() *HandlerOptions

HandlerOptions returns a copy of the current handler options. This will never return nil.

func (*Handler) Named

func (h *Handler) Named(name string) slog.Handler

Named is a convenience for `h.WithAttrs([]slog.Attr{slog.String(LoggerKey, name)})`.

func (*Handler) Out

func (h *Handler) Out() io.Writer

func (*Handler) SetHandlerOptions

func (h *Handler) SetHandlerOptions(opts *HandlerOptions)

SetHandlerOptions sets the options passed to HandlerFn when sink handlers are created. This triggers rebuilding all the sink handlers with the new opts, so affects on logging will apply immediately.

func (*Handler) SetOut

func (h *Handler) SetOut(w io.Writer)

SetOut sets the output writer passed to HandlerFn when sink handlers are created. This triggers rebuilding all the sink handlers with the new opts, so affects on logging will apply immediately.

func (*Handler) WithAttrs

func (h *Handler) WithAttrs(attrs []slog.Attr) slog.Handler

func (*Handler) WithGroup

func (h *Handler) WithGroup(name string) slog.Handler

type HandlerFn

type HandlerFn func(name string, w io.Writer, opts *slog.HandlerOptions) slog.Handler

HandlerFn is a constructor for slog handlers. The function should return a slog.Handler configured with the given writer and options. `w` and `opts` will never be nil.

`name` is the name of the logger for which the handler is being created, e.g. via flume.New("<name>") or logger.With(flume.LoggerKey, "<name>"). This parameter may be used to return different handlers or different middleware for particular loggers.

func JSONHandlerFn

func JSONHandlerFn() HandlerFn

JSONHandlerFn is shorthand for LookupHandlerFn("json"). Will never be nil.

func LookupHandlerFn

func LookupHandlerFn(name string) HandlerFn

LookupHandlerFn looks for a handler registered with the given name. Registered handlers are stored in an internal, package level map, which is initialized with some built-in handlers. Handlers can be added or replaced via RegisterHandlerFn.

Returns nil if name is not found.

LookupHandlerFn is used when unmarshaling HandlerOptions from json, to resolve the "handler" property to a handler function.

func NoopHandlerFn

func NoopHandlerFn() HandlerFn

NoopHandlerFn is shorthand for LookupHandlerFn("noop"). Will never be nil.

func TermColorHandlerFn

func TermColorHandlerFn() HandlerFn

TermColorHandlerFn is shorthand for LookupHandlerFn("term-color"). Will never be nil.

func TermHandlerFn

func TermHandlerFn() HandlerFn

TermHandlerFn is shorthand for LookupHandlerFn("term"). Will never be nil.

func TextHandlerFn

func TextHandlerFn() HandlerFn

TextHandlerFn is shorthand for LookupHandlerFn("text"). Will never be nil.

type HandlerOptions

type HandlerOptions struct {
	// default level for all loggers, defaults to slog.LevelInfo
	Level slog.Leveler
	// per-logger levels
	Levels map[string]slog.Leveler
	// add source to log records
	AddSource bool
	// replace attributes
	ReplaceAttrs []func(groups []string, a slog.Attr) slog.Attr
	// If set, will be called to construct handler instances.
	// Defaults to TextHandlerFn()
	HandlerFn HandlerFn
	// middleware applied to all sinks
	Middleware []Middleware
}

func DevDefaults

func DevDefaults() *HandlerOptions

func (*HandlerOptions) Clone

func (o *HandlerOptions) Clone() *HandlerOptions

func (*HandlerOptions) UnmarshalJSON

func (o *HandlerOptions) UnmarshalJSON(bytes []byte) error

type Levels

type Levels map[string]slog.Leveler

func (*Levels) MarshalText

func (l *Levels) MarshalText() ([]byte, error)

func (*Levels) UnmarshalText

func (l *Levels) UnmarshalText(text []byte) error

type Middleware

type Middleware interface {
	Apply(slog.Handler) slog.Handler
}

type MiddlewareFn

type MiddlewareFn func(slog.Handler) slog.Handler

MiddlewareFn adapts a function to the Middleware interface.

func (MiddlewareFn) Apply

func (f MiddlewareFn) Apply(h slog.Handler) slog.Handler

type ReplaceAttrsMiddleware

type ReplaceAttrsMiddleware struct {
	// If SkipRecord is set, it will be called in the Handle function.  If it returns
	// false, all attr processing will be skipped.
	//
	// This may be used if attr processing is conditional on some aspect of the record,
	// like the level.  Note that attrs passed to WithAttrs() are always processed.
	SkipRecord func(slog.Record) bool

	// If true, built-in attributes (message, level, and time) will not be processed.
	SkipBuiltins bool
	// contains filtered or unexported fields
}

func ReplaceAttrs

func ReplaceAttrs(fns ...func([]string, slog.Attr) slog.Attr) *ReplaceAttrsMiddleware

ReplaceAttrs is middleware which adds ReplaceAttr support to other Handlers which don't natively have it. Because this can only act on the slog.Record as it passes through the middleware, it has limitations regarding the built-in fields:

  • slog.SourceKey: skipped
  • slog.MessageKey: ignores changes to the key name. If the returned value is empty, the message will be set to the value `<nil>`. Non-string values are coerced to a string like `fmt.Print`
  • slog.TimeKey: ignores changes to the key name. The slog.Value returned must either be empty, or a slog.KindTime. Other values will be ignored.
  • slog.LevelKey: ignores changes to the key name. The slog.Value returned must either be empty, or be a slog.KindAny containing a slog.Level value. Other values will be ignored.

The middleware may be further configured to skip processing built-in attributes, or skipping processing for particular records.

func (*ReplaceAttrsMiddleware) Apply

func (*ReplaceAttrsMiddleware) Enabled

func (r *ReplaceAttrsMiddleware) Enabled(ctx context.Context, level slog.Level) bool

func (*ReplaceAttrsMiddleware) Handle

func (r *ReplaceAttrsMiddleware) Handle(ctx context.Context, record slog.Record) error

func (*ReplaceAttrsMiddleware) WithAttrs

func (r *ReplaceAttrsMiddleware) WithAttrs(attrs []slog.Attr) slog.Handler

func (*ReplaceAttrsMiddleware) WithGroup

func (r *ReplaceAttrsMiddleware) WithGroup(name string) slog.Handler

type SimpleMiddlewareFn

type SimpleMiddlewareFn func(ctx context.Context, record slog.Record, next slog.Handler) error

SimpleMiddlewareFn defines a simple middleware function which intercepts *slog.Handler.Handle() calls. SimpleMiddlewareFn adapts this to the Middleware interface.

func (SimpleMiddlewareFn) Apply

Apply implements Middleware

type SlogAdapter

type SlogAdapter struct {
	// contains filtered or unexported fields
}

func NewSlogAdapter

func NewSlogAdapter(l FlumeV1Logger) *SlogAdapter

NewSlogAdapter create an adapter which implements SlogLogger, and translates those calls to an old flumev1/yugolog instance.

func (*SlogAdapter) Debug

func (s *SlogAdapter) Debug(msg string, args ...any)

func (*SlogAdapter) DebugContext

func (s *SlogAdapter) DebugContext(_ context.Context, msg string, args ...any)

func (*SlogAdapter) Enabled

func (s *SlogAdapter) Enabled(_ context.Context, l slog.Level) bool

func (*SlogAdapter) Error

func (s *SlogAdapter) Error(msg string, args ...any)

func (*SlogAdapter) ErrorContext

func (s *SlogAdapter) ErrorContext(_ context.Context, msg string, args ...any)

func (*SlogAdapter) Info

func (s *SlogAdapter) Info(msg string, args ...any)

func (*SlogAdapter) InfoContext

func (s *SlogAdapter) InfoContext(_ context.Context, msg string, args ...any)

func (*SlogAdapter) Log

func (s *SlogAdapter) Log(_ context.Context, level slog.Level, msg string, args ...any)

func (*SlogAdapter) Warn

func (s *SlogAdapter) Warn(msg string, args ...any)

func (*SlogAdapter) WarnContext

func (s *SlogAdapter) WarnContext(_ context.Context, msg string, args ...any)

type SlogLogger

type SlogLogger interface {
	Enabled(ctx context.Context, l slog.Level) bool
	Log(ctx context.Context, level slog.Level, msg string, args ...any)

	Debug(msg string, args ...any)
	DebugContext(ctx context.Context, msg string, args ...any)

	Info(msg string, args ...any)
	InfoContext(ctx context.Context, msg string, args ...any)
	Warn(msg string, args ...any)
	WarnContext(ctx context.Context, msg string, args ...any)

	Error(msg string, args ...any)
	ErrorContext(ctx context.Context, msg string, args ...any)
}

SlogLogger describes most of the commonly used parts of the *slog.Logger API. *slog.Logger implements this interface natively. The intent is make your code depend on this interface, instead of directly on *slog.Logger. This will let you inject either a *slog.Logger or a *SlogAdapter.

Directories

Path Synopsis
Package flumetest configures flume to integrate with golang tests.
Package flumetest configures flume to integrate with golang tests.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL