Documentation
¶
Overview ¶
Package log provides a wrapped zap logger interface for microservices to use, and also a simple Wrapper interface to be used by other Baseplate.go packages.
For zap logger related features, we provide both a global logger which can be used by top level functions, and a way to attach logger with additional info (e.g. trace id) to context object and reuse. When you need to use the zap logger and you have a context object, you should use the logger attached to the context, like:
log.C(ctx).Errorw("Something went wrong!", "err", err)
But if you don't have a context object, instead of creating one to use logger, you should use the global one:
log.Errorw("Something went wrong!", "err", err)
Index ¶
- Constants
- Variables
- func Attach(ctx context.Context, args AttachArgs) context.Context
- func C(ctx context.Context) *zap.SugaredLogger
- func CapitalLevelEncoder(l zapcore.Level, enc zapcore.PrimitiveArrayEncoder)
- func DPanic(args ...interface{})
- func DPanicf(template string, args ...interface{})
- func DPanicw(msg string, keysAndValues ...interface{})
- func Debug(args ...interface{})
- func Debugf(template string, args ...interface{})
- func Debugw(msg string, keysAndValues ...interface{})
- func Error(args ...interface{})
- func ErrorWithSentry(ctx context.Context, msg string, err error, keysAndValues ...interface{})
- func Errorf(template string, args ...interface{})
- func Errorw(msg string, keysAndValues ...interface{})
- func Fatal(args ...interface{})
- func Fatalf(template string, args ...interface{})
- func Fatalw(msg string, keysAndValues ...interface{})
- func FullCallerEncoder(caller zapcore.EntryCaller, enc zapcore.PrimitiveArrayEncoder)
- func Info(args ...interface{})
- func Infof(template string, args ...interface{})
- func Infow(msg string, keysAndValues ...interface{})
- func InitFromConfig(cfg Config)
- func InitLogger(logLevel Level)
- func InitLoggerJSON(logLevel Level)
- func InitLoggerWithConfig(logLevel Level, cfg zap.Config) error
- func InitSentry(cfg SentryConfig) (io.Closer, error)
- func JSONTimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder)
- func NopWrapper(ctx context.Context, msg string)
- func Panic(args ...interface{})
- func Panicf(template string, args ...interface{})
- func Panicw(msg string, keysAndValues ...interface{})
- func SentryBeforeSendSwapExceptionTypeAndValue(event *sentry.Event, hint *sentry.EventHint) *sentry.Event
- func ShortCallerEncoder(caller zapcore.EntryCaller, enc zapcore.PrimitiveArrayEncoder)
- func Sync() error
- func TimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder)
- func Warn(args ...interface{})
- func Warnf(template string, args ...interface{})
- func Warnw(msg string, keysAndValues ...interface{})
- func With(args ...interface{}) *zap.SugaredLogger
- func WrapToThriftLogger(w Wrapper) thrift.Loggerdeprecated
- type AttachArgs
- type Config
- type Counter
- type KitWrapper
- type Level
- type SentryConfig
- type Wrapper
- type ZapWrapperArgs
Examples ¶
Constants ¶
const DefaultSentryFlushTimeout = time.Second * 2
DefaultSentryFlushTimeout is the timeout used to call sentry.Flush().
const RFC3339Nano = "ts=2006-01-02T15:04:05.000000Z"
RFC3339Nano is a time format for TimeEncoder
Variables ¶
var ( VersionLogKey = "v" Version string )
Version is the version tag value to be added to the global logger.
If it's changed to non-empty value before the calling of Init* functions (InitFromConfig/InitLogger/InitLoggerJSON/InitLoggerWithConfig/InitSentry), the global logger initialized will come with a tag of VersionKey("v"), added to every line of logs. For InitSentry the global sentry will also be initialized with a tag of "version".
In order to use it, either set it in your main function early, before calling InitLogger* functions, with the value coming from flag/config file/etc.. For example:
func main() {
log.Version = *flagVersion
log.InitLoggerJSON(log.Level(*logLevel))
// ...
}
Or just "stamp" it during build time, by passing additional ldflags to go build command. For example:
go build -ldflags "-X github.com/reddit/baseplate.go/log.Version=$(git rev-parse HEAD)"
Change its value after calling Init* functions will have no effects, unless you call Init* functions again.
Starting from go 1.18, if Version is not stamped at compile time, we'll try to fill it from runtime/debug.ReadBuildInfo, if available.
var ErrSentryFlushFailed = errors.New("log: sentry flushing failed")
ErrSentryFlushFailed is an error could be returned by the Closer returned by InitSentry, to indicate that the sentry flushing failed.
Functions ¶
func Attach ¶ added in v0.2.0
func Attach(ctx context.Context, args AttachArgs) context.Context
Attach attaches a logger and sentry hub with data extracted from args into the context object.
func C ¶ added in v0.2.0
func C(ctx context.Context) *zap.SugaredLogger
C is short for Context.
It extract the logger attached to the current context object, and fallback to the global logger if none is found.
When you have a context object and want to do logging, you should always use this one instead of the global one. For example:
log.C(ctx).Errorw("Something went wrong!", "err", err)
The return value is guaranteed to be non-nil.
func CapitalLevelEncoder ¶
func CapitalLevelEncoder(l zapcore.Level, enc zapcore.PrimitiveArrayEncoder)
CapitalLevelEncoder adds logger level in uppercase
func DPanic ¶
func DPanic(args ...interface{})
DPanic uses fmt.Sprint to construct and log a message.
In development, the logger then panics. (See DPanicLevel for details.)
func DPanicf ¶
func DPanicf(template string, args ...interface{})
DPanicf uses fmt.Sprintf to log a templated message.
In development, the logger then panics. (See DPanicLevel for details.)
func DPanicw ¶
func DPanicw(msg string, keysAndValues ...interface{})
DPanicw logs a message with some additional context.
In development, the logger then panics. (See DPanicLevel for details.)
The variadic key-value pairs are treated as they are in With.
func Debugf ¶
func Debugf(template string, args ...interface{})
Debugf uses fmt.Sprintf to log a templated message.
func Debugw ¶
func Debugw(msg string, keysAndValues ...interface{})
Debugw logs a message with some additional context.
The variadic key-value pairs are treated as they are in With.
When debug-level logging is disabled, this is much faster than
With(keysAndValues).Debug(msg)
func ErrorWithSentry ¶
ErrorWithSentry logs a message with some additional context, then sends the error to Sentry.
The variadic key-value pairs are treated as they are in With. and will also be sent to sentry. Note that zap.Field is not supported here and will be ignored while sending to sentry (but they will be logged to error log).
If a sentry hub is attached to the context object passed in (it will be if the context object is from baseplate hooked request context), that hub will be used to do the reporting. Otherwise the global sentry hub will be used instead.
func Errorf ¶
func Errorf(template string, args ...interface{})
Errorf uses fmt.Sprintf to log a templated message.
func Errorw ¶
func Errorw(msg string, keysAndValues ...interface{})
Errorw logs a message with some additional context.
The variadic key-value pairs are treated as they are in With.
func Fatal ¶
func Fatal(args ...interface{})
Fatal uses fmt.Sprint to construct and log a message, then calls os.Exit.
func Fatalf ¶
func Fatalf(template string, args ...interface{})
Fatalf uses fmt.Sprintf to log a templated message, then calls os.Exit.
func Fatalw ¶
func Fatalw(msg string, keysAndValues ...interface{})
Fatalw logs a message with some additional context, then calls os.Exit.
The variadic key-value pairs are treated as they are in With.
func FullCallerEncoder ¶
func FullCallerEncoder(caller zapcore.EntryCaller, enc zapcore.PrimitiveArrayEncoder)
FullCallerEncoder serializes a caller in /full/path/to/package/file:line format.
func Infof ¶
func Infof(template string, args ...interface{})
Infof uses fmt.Sprintf to log a templated message.
func Infow ¶
func Infow(msg string, keysAndValues ...interface{})
Infow logs a message with some additional context.
The variadic key-value pairs are treated as they are in With.
func InitFromConfig ¶
func InitFromConfig(cfg Config)
InitFromConfig initializes the log package using the given Config and JSON logger.
func InitLogger ¶
func InitLogger(logLevel Level)
InitLogger provides a quick way to start or replace the global logger.
func InitLoggerJSON ¶
func InitLoggerJSON(logLevel Level)
InitLoggerJSON initializes global logger with full json format.
The JSON format is also compatible with logdna's ingestion format: https://docs.logdna.com/docs/ingestion
func InitLoggerWithConfig ¶
InitLoggerWithConfig provides a quick way to start or replace the global logger.
Pass in a cfg to provide a logger with custom setting.
This function also wraps the default zap core to convert all int64 and uint64 fields to strings, to prevent the loss of precision by json log ingester. As a result, some of the cfg might get lost during this wrapping, namely OutputPaths and ErrorOutputPaths.
func InitSentry ¶
func InitSentry(cfg SentryConfig) (io.Closer, error)
InitSentry initializes sentry reporting.
The io.Closer returned calls sentry.Flush with SentryFlushTimeout. If it returns an error, that error is guaranteed to wrap ErrSentryFlushFailed.
You can also just call sentry.Init, which provides even more customizations. This function is provided to do the customizations we care about the most, and to provide a Closer to be more consistent with other baseplate packages.
When cfg.ServerName is non-empty, this function also sets hostname tag to the value reported by the kernel (when cfg.ServerName is empty server_name tag will be the hostname).
func JSONTimeEncoder ¶
func JSONTimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder)
JSONTimeEncoder encodes time in RFC3339Nano without extra information.
It's suitable to be used in the full JSON format.
func NopWrapper ¶
NopWrapper is a Wrapper implementation that does nothing.
In most cases you don't need to use it directly. The zero value of log.Wrapper is essentially a NopWrapper.
func Panic ¶
func Panic(args ...interface{})
Panic uses fmt.Sprint to construct and log a message, then panics.
func Panicf ¶
func Panicf(template string, args ...interface{})
Panicf uses fmt.Sprintf to log a templated message, then panics.
func Panicw ¶
func Panicw(msg string, keysAndValues ...interface{})
Panicw logs a message with some additional context, then panics.
The variadic key-value pairs are treated as they are in With.
func SentryBeforeSendSwapExceptionTypeAndValue ¶ added in v0.9.0
func SentryBeforeSendSwapExceptionTypeAndValue(event *sentry.Event, hint *sentry.EventHint) *sentry.Event
SentryBeforeSendSwapExceptionTypeAndValue is a sentry.BeforeSend implementation that swaps the error message and error type reported to sentry.
See 1 for context on why doing this might be a good idea for your service.
This is not enabled by default, as it makes sentry no longer cluster errors with same type but slightly different error message as the same error. If you want to use this feature, set SentryConfig.BeforeSend to this function. If you have other things you want to do in your BeforeSend, you can call this function in your BeforeSend implementation instead. e.g.:
sentryConfig.BeforeSend = func(event *sentry.Event, hint *sentry.EventHint) *sentry.Event {
// Do other things you want to do here.
// Also do the swapping.
return log.SentryBeforeSendSwapExceptionTypeAndValue(event, hint)
}
func ShortCallerEncoder ¶
func ShortCallerEncoder(caller zapcore.EntryCaller, enc zapcore.PrimitiveArrayEncoder)
ShortCallerEncoder serializes a caller in package/file:line format, trimming all but the final directory from the full path.
func TimeEncoder ¶
func TimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder)
TimeEncoder is customized to add ts in the front
func Warnf ¶
func Warnf(template string, args ...interface{})
Warnf uses fmt.Sprintf to log a templated message.
func Warnw ¶
func Warnw(msg string, keysAndValues ...interface{})
Warnw logs a message with some additional context.
The variadic key-value pairs are treated as they are in With.
func With ¶
func With(args ...interface{}) *zap.SugaredLogger
With wraps around the underline logger's With.
func WrapToThriftLogger
deprecated
added in
v0.6.0
Types ¶
type AttachArgs ¶ added in v0.2.0
AttachArgs are used to create loggers and sentry hubs to be attached to context object with pre-filled key-value pairs.
All zero value fields will be ignored and only non-zero values will be attached.
AdditionalPairs are provided to add any free form, additional key-value pairs you want to attach to all logs and sentry reports from the same context object.
type Config ¶
type Config struct {
// Level is the log level you want to set your service to.
Level Level `yaml:"level"`
}
Config is the confuration struct for the log package.
Can be deserialized from YAML.
type Counter ¶ added in v0.9.6
type Counter interface {
Add(float64)
}
Counter is a minimal interface for a counter.
This is implemented by both prometheus counter and statsd counter from metricsbp.
type KitWrapper ¶
KitWrapper is a type that implements go-kit/log.Logger interface with zap logger.
func KitLogger ¶
func KitLogger(level Level) KitWrapper
KitLogger returns a go-kit compatible logger.
func (KitWrapper) Log ¶
func (w KitWrapper) Log(keyvals ...interface{}) error
Log implements go-kit/log.Logger interface.
type Level ¶
type Level string
Level is the verbose representation of log level
const ( NopLevel Level = "nop" DebugLevel Level = "debug" InfoLevel Level = "info" WarnLevel Level = "warn" ErrorLevel Level = "error" PanicLevel Level = "panic" FatalLevel Level = "fatal" // This will have the same effect as nop but slower ZapNopLevel zapcore.Level = zapcore.FatalLevel + 1 )
Enums for Level
func (Level) ToZapLevel ¶
ToZapLevel converts Level to a zap acceptable atomic level struct
type SentryConfig ¶
type SentryConfig struct {
// The Sentry DSN to use.
// If empty, SENTRY_DSN environment variable will be used instead.
// If that's also empty, then all sentry operations will be nop.
DSN string `yaml:"dsn"`
// SampleRate between 0 and 1, default is 1.
SampleRate *float64 `yaml:"sampleRate"`
// The name of your service.
//
// By default sentry extracts hostname reported by the kernel for this field.
// Set it to non-empty to override that
// (and hostname tag will be used to report hostname automatically).
ServerName string `yaml:"serverName"`
// An environment string like "prod", "staging".
Environment string `yaml:"environment"`
// List of regexp strings that will be used to match against event's message
// and if applicable, caught errors type and value.
// If the match is found, then a whole event will be dropped.
IgnoreErrors []string `yaml:"ignoreErrors,omitempty"`
// FlushTimeout is the timeout to be used to call sentry.Flush when closing
// the Closer returned by InitSentry.
// If <=0, DefaultSentryFlushTimeout will be used.
FlushTimeout time.Duration `yaml:"flushTimeout"`
// BeforeSend is a callback modifier before emitting an event to Sentry.
BeforeSend func(event *sentry.Event, hint *sentry.EventHint) *sentry.Event `yaml:"-"`
}
SentryConfig is the config to be passed into InitSentry.
All fields are optional.
type Wrapper ¶
Wrapper defines a simple interface to wrap logging functions.
As principles, library code should:
1. Not do any logging. The library code should communicate errors back to the caller, and let the caller decide how to deal with them (log them, ignore them, panic, etc.)
2. In some rare cases, 1 is not possible, for example the error might happen in a background goroutine. In those cases some logging is necessary, but those should be kept at minimal, and the library code should provide control to the caller on how to do those logging.
This interface is meant to solve Principle 2 above. In reality we might actually use different logging libraries in different services, and they are not always compatible with each other. Wrapper is a simple common ground that it's easy to wrap whatever logging library we use into.
With that in mind, this interface should only be used by library code, when the case satisfies all of the following 3 criteria:
1. A bad thing happened.
2. This is unexpected. For expected errors, the library should either handle it by itself (e.g. retry), or communicate it back to the caller and let them handle it.
3. This is also recoverable. Unrecoverable errors should also be communicated back to the caller to handle.
Baseplate services should use direct logging functions for their logging needs, instead of using Wrapper interface.
For production code using baseplate libraries, Baseplate services should use ErrorWithSentryWrapper in most cases, as whenever the Wrapper is called that's something bad and unexpected happened and the service owner should be aware of. Non-Baseplate services should use error level in whatever logging library they use.
metricsbp.LogWrapper also provides an implementation that emits metrics when it's called. It can be wrapped on top of other log.Wrapper implementations.
For unit tests of library code using Wrapper, TestWrapper is provided that would fail the test when Wrapper is called.
Not all Wrapper implementations take advantage of context object passed in, but the caller should always pass it into Wrapper if they already have one, or just use context.Background() if they don't have one.
var DefaultWrapper Wrapper = ErrorWithSentryWrapper()
DefaultWrapper defines the default one to be used in log.Wrapper.
It affects two places:
1. When using nil-safe calls on log.Wrapper on a nil log.Wrapper.
2. When unmarshaling from text (yaml) and the text is empty.
func CounterWrapper ¶ added in v0.9.6
CounterWrapper returns a Wrapper implementation that increases counter by 1 then calls delegate to log the message.
Please note that it's not possible to deserialize this Wrapper directly from yaml, so you usually need to override it in your main function, after baseplate.ParseConfigYAML call, for example:
// a global variable
var tracingFailures = promauto.NewCounter(prometheus.CounterOpts{
Namespace: "myservice",
Subsystem: "tracing",
Name: "failures_total",
Help: "Total number of failures when sending tracing spans to the sidecar",
})
// in main
if err := baseplate.ParseConfigYAML(&cfg); err != nil {
log.Fatal(err)
}
cfg.Config.Tracing.Logger = log.CounterWrapper(
cfg.Config.Tracing.Logger, // delegate
tracingFailures, // counter
}
func ErrorWithSentryWrapper ¶
func ErrorWithSentryWrapper() Wrapper
ErrorWithSentryWrapper is a Wrapper implementation that both use Zap logger to log at error level, and also send the message to Sentry.
In most cases this should be the one used to pass into Baseplate.go libraries expecting a log.Wrapper. If the service didn't configure sentry, then this wrapper is essentially the same as log.ZapWrapper(log.ErrorLevel).
Note that this should not be used as the logger set into thrift.TSimpleServer, as that would use the logger to log network I/O errors, which would be too spammy to be sent to Sentry. For this reason, it's returning a Wrapper instead of being a Wrapper itself, thus forcing an extra typecasting to be used as a thrift.Logger.
func StdWrapper ¶
StdWrapper wraps stdlib log package into a Wrapper.
func TestWrapper ¶
TestWrapper is a wrapper can be used in test codes.
It fails the test when called.
func ZapWrapper ¶
func ZapWrapper(args ZapWrapperArgs) Wrapper
ZapWrapper wraps zap log package into a Wrapper.
func (Wrapper) Log ¶ added in v0.3.0
Log is the nil-safe way of calling a log.Wrapper.
If w is nil, DefaultWrapper will be used instead.
func (Wrapper) MarshalYAML ¶ added in v0.9.19
MarshalYAML implements yaml.Marshaler (both v2 and v3).
It only support nil values, as it's almost impossible to detect which non-nil value it really is to marshal to some meaningful value that can be unmarshaled.
func (Wrapper) ToThriftLogger
deprecated
added in
v0.6.0
func (*Wrapper) UnmarshalText ¶ added in v0.7.1
UnmarshalText implements encoding.TextUnmarshaler.
It makes Wrapper possible to be used directly in yaml and other config files.
Please note that this currently only support limited implementations:
- empty: DefaultWrapper.
- "nop": NopWrapper.
- "std": StdWrapper with default stdlib logger (log.New(os.Stderr, "", log.LstdFlags)).
- "zap": ZapWrapper on default level (Info) with no kv pairs.
- "zap:level:k1=v1,k2=v2...": ZapWrapper with given level and kv pairs, with the ":k=v..." part being optional. For example "zap:error" means ZapWrapper on Error level with no kv pairs, "zap:info:key1=value1" means ZapWrapper on Info level with "key1":"value1" pair.
- "sentry": ErrorWithSentryWrapper.
See the example on how to extend it to support other implementations.
Example ¶
This example demonstrates how to write your own type to "extend" log.Wrapper.UnmarshalText to add other implementations.
package main
import (
"bytes"
"context"
"encoding"
"fmt"
"strings"
"gopkg.in/yaml.v2"
"github.com/reddit/baseplate.go/log"
"github.com/reddit/baseplate.go/metricsbp"
)
// ExtendedLogWrapper extends log.Wrapper to support metricsbp.LogWrapper.
type ExtendedLogWrapper struct {
log.Wrapper
}
var st = metricsbp.NewStatsd(
context.Background(),
metricsbp.Config{
// This is to make sure that when we read the metricsbp statsd buffer out
// manually later in the example they are not already sent to a blackhole
// sink.
// This is NOT needed in production code as in production code the sink
// shall be configured as the actual statsd collector and not read out
// manually like in this example.
BufferInMemoryForTesting: true,
},
)
// UnmarshalText implements encoding.TextUnmarshaler.
//
// In addition to the implementations log.Wrapper.UnmarshalText supports, it
// added supports for:
//
// - "counter:metrics:logger": metricsbp.LogWrapper, with "metrics" being the
// name of the counter metrics and "logger" being the underlying logger.
func (e *ExtendedLogWrapper) UnmarshalText(text []byte) error {
const counterPrefix = "counter:"
if s := string(text); strings.HasPrefix(s, counterPrefix) {
parts := strings.SplitN(s, ":", 3)
if len(parts) != 3 {
return fmt.Errorf("unsupported log.Wrapper config: %q", text)
}
var w log.Wrapper
if err := w.UnmarshalText([]byte(parts[2])); err != nil {
return err
}
e.Wrapper = metricsbp.LogWrapper(metricsbp.LogWrapperArgs{
Counter: parts[1],
Wrapper: w,
Statsd: st,
})
return nil
}
return e.Wrapper.UnmarshalText(text)
}
func (e ExtendedLogWrapper) ToLogWrapper() log.Wrapper {
return e.Wrapper
}
var _ encoding.TextUnmarshaler = (*ExtendedLogWrapper)(nil)
// This example demonstrates how to write your own type to "extend"
// log.Wrapper.UnmarshalText to add other implementations.
func main() {
const (
invalid = `logger: fancy`
counterOnly = `logger: "counter:foo.bar.count:nop"`
)
var data struct {
Logger ExtendedLogWrapper `yaml:"logger"`
}
fmt.Printf(
"This is an invalid config: %s, err: %v\n",
invalid,
yaml.Unmarshal([]byte(invalid), &data),
)
fmt.Printf(
"This is an counter-only config: %s, err: %v\n",
counterOnly,
yaml.Unmarshal([]byte(counterOnly), &data),
)
data.Logger.Log(context.Background(), "Hello, world!")
var buf bytes.Buffer
st.WriteTo(&buf)
fmt.Printf("Counter: %s", buf.String())
}
Output: This is an invalid config: logger: fancy, err: unsupported log.Wrapper config: "fancy" This is an counter-only config: logger: "counter:foo.bar.count:nop", err: <nil> Counter: foo.bar.count:1.000000|c
type ZapWrapperArgs ¶ added in v0.9.0
ZapWrapperArgs defines the args used in ZapWrapper.
Source Files