README
¶
Jessy is a high-performance JSON library for Go, focused on speed and reduced memory usage. It's a drop-in replacement for encoding/json and doesn't have dependencies
Before diving into the details, let's start with some metrics
Table of fastest big map encoding for demonstration memory usage
| ns/op | allocation bytes | allocation times | |
|---|---|---|---|
| jessy ✅ | 232381883 | 321 | 1 |
| jessy-pretty ✅ | 272061406 | 382 | 1 |
| sonic | 748173167 | 489671300 | 40 |
| jettison | 292332917 | 510 | 3 |
| jsoniter | 542650438 | 559997264 | 5500005 |
| gojson | 344189542 | 337964296 | 100038 |
| encoding/json | 517449250 | 95207116 | 2700004 |
Marshal in compatibility mode
Fastest encode big map of 100k medium structs
Philosophy
- Use reflection only once during the initial type processing
- Perform later processing using only direct unsafe memory access
- Minimize actions, checks, and conditions during repeated traversals
- During the initial type processing, create a unique encoder for each field and nested type/structure
- Always consider memory usage and speed
- Benchmark every change to ensure performance
- Fight for every nanosecond
API
// This is just a small part of all the available functions
// Marshal with encoding/json compatibility
func Marshal(value any) ([]byte, error)
// Marshal with encoding/json compatibility and \t indents
func MarshalPretty(value any) ([]byte, error)
// Marshal with fair indents (will be extra allocations, recommend use Pretty)
func MarshalIndent(value any, prefix, indent string) ([]byte, error)
// Fastest marshal without compatibility (e.g. unsorted maps)
func MarshalFast(value any) ([]byte, error)
// Fastest marshal with \t indents
func MarshalPrettyFast(value any) ([]byte, error)
// Marshal with custom parameters
func MarshalFlags(value any, flags Flags) ([]byte, error)
// All next append functions perform the same marshal, but use the provided buffer to avoid allocations.
// Highly recommended to use!
// Append json with encoding/json compatibility
func Append(dst []byte, value any) ([]byte, error)
// Fastest append json without compatibility
func AppendFast(dst []byte, value any) ([]byte, error)
// Append json with custom parameters
func AppendFlags(dst []byte, value any, flags Flags) ([]byte, error)
// You can also pre-cache in advance (not required)
func MarshalPrecache(value any, flags Flags)
func MarshalPrecacheFor[T any](flags Flags)
// Usually encoding/json Unmarshal
func Unmarshal(data []byte, v any) error
// Fast encoding/json unmarshal without checks
func UnmarshalTrusted(data []byte, v any) error
More zeroalloc marshal
You can implement special AppendMarshaler/AppendTextMarshaler interface for zeroalloc some structs
type AppendMarshaler interface {
AppendJSON(dst []byte) (newDst []byte, err error)
}
type AppendTextMarshaler interface {
AppendText(dst []byte) (newDst []byte, err error)
}
Custom marshal encoder
No matter how much we want to marshal a structure without memory allocations, sometimes our structures contain types from other libraries that we can't change.
The current library provides a way to handle this!
type UnsafeEncoder func(dst []byte, value unsafe.Pointer) ([]byte, error)
type ValueEncoder[T any] func(dst []byte, value T) ([]byte, error)
func AddUnsafeEncoder[T any](encoder func(flags Flags) UnsafeEncoder)
func AddValueEncoder[T any](encoder func(flags Flags) ValueEncoder[T])
Using example
import "github.com/avpetkun/jessy-go"
type MyType [10]byte
jessy.AddValueEncoder(func(flags Flags) jessy.ValueEncoder[MyType] {
return func(dst []byte, v MyType) ([]byte, error) {
dst = append(dst, '"')
dst = appendHex(dst, v[:])
dst = append(dst, '"')
return dst, nil
}
})
Drop-in replacement
Replace
import (
"encoding/json"
)
json.Marshal(data)
with
import (
json "github.com/avpetkun/jessy-go"
)
json.Marshal(data)
Hash
You can get fnv hash by all struct values
import "github.com/avpetkun/jessy-go"
hash, err := jessy.Hash(struct{A int}{123})
Features
In addition to the mentioned benefits, the library also:
- Can marshal complex numbers
- Can marshal maps with any key type
TODO
- Extend the philosophy to the Unmarshal process
- Optimize memory usage for map keys sorting
- Add more code comments
How to get
go get github.com/avpetkun/jessy-go
Contribution Welcomed !
Documentation
¶
Index ¶
- func AddUnsafeEncoder[T any](encoder func(flags Flags) UnsafeEncoder)
- func AddValueEncoder[T any](encoder func(flags Flags) ValueEncoder[T])
- func Append(dst []byte, value any) ([]byte, error)
- func AppendFast(dst []byte, value any) ([]byte, error)
- func AppendFlags(dst []byte, value any, flags Flags) ([]byte, error)
- func AppendIndent(dst []byte, value any, prefix, indent string) ([]byte, error)
- func AppendIndentFast(dst []byte, value any, prefix, indent string) ([]byte, error)
- func AppendIndentFlags(dst []byte, value any, flags Flags, prefix, indent string) (data []byte, err error)
- func AppendPretty(dst []byte, value any) ([]byte, error)
- func AppendPrettyFast(dst []byte, value any) ([]byte, error)
- func Compact(dst *bytes.Buffer, src []byte) (err error)
- func HTMLEscape(dst *bytes.Buffer, src []byte)
- func Hash(value any) (hashSum uint64, err error)
- func Indent(dst *bytes.Buffer, src []byte, prefix, indent string) (err error)
- func Marshal(value any) ([]byte, error)
- func MarshalFast(value any) ([]byte, error)
- func MarshalFlags(value any, flags Flags) (dst []byte, err error)
- func MarshalIndent(value any, prefix, indent string) ([]byte, error)
- func MarshalIndentFast(value any, prefix, indent string) ([]byte, error)
- func MarshalIndentFlags(value any, flags Flags, prefix, indent string) ([]byte, error)
- func MarshalPrecache(value any, flags Flags)
- func MarshalPrecacheFor[T any](flags Flags)
- func MarshalPretty(value any) ([]byte, error)
- func MarshalPrettyFast(value any) ([]byte, error)
- func ResetEncodersCache()
- func SetMarshalMaxDeep(deep int)
- func Unmarshal(data []byte, v any) error
- func UnmarshalTrusted(data []byte, v any) error
- func Valid(data []byte) bool
- type AppendMarshaler
- type AppendTextMarshaler
- type Decoder
- type Encoder
- func (e *Encoder) Encode(value any) (err error)
- func (e *Encoder) EncodeRaw(value any) (data []byte, err error)
- func (e *Encoder) Grow(size int)
- func (e *Encoder) GrowIndent(size int)
- func (e *Encoder) Reset(w io.Writer)
- func (e *Encoder) SetEscapeHTML(on bool)
- func (e *Encoder) SetFastestFlags()
- func (e *Encoder) SetFlags(flags Flags)
- func (e *Encoder) SetIndent(prefix, indent string)
- func (e *Encoder) SetPrettyFlags(on bool)
- func (e *Encoder) SetStandardFlags()
- type Flags
- type Marshaler
- type Number
- type RawMessage
- type StructField
- type TextMarshaler
- type TextUnmarshaler
- type Unmarshaler
- type UnsafeEncoder
- type ValueEncoder
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AddUnsafeEncoder ¶
func AddUnsafeEncoder[T any](encoder func(flags Flags) UnsafeEncoder)
func AddValueEncoder ¶
func AddValueEncoder[T any](encoder func(flags Flags) ValueEncoder[T])
func AppendIndentFast ¶
func AppendIndentFlags ¶
func Compact ¶
Compact appends to dst the JSON-encoded src with insignificant space characters elided.
func HTMLEscape ¶
HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029 characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029 so that the JSON will be safe to embed inside HTML <script> tags. For historical reasons, web browsers don't honor standard HTML escaping within <script> tags, so an alternative JSON encoding must be used.
func Indent ¶
Indent appends to dst an indented form of the JSON-encoded src. Each element in a JSON object or array begins on a new, indented line beginning with prefix followed by one or more copies of indent according to the indentation nesting. The data appended to dst does not begin with the prefix nor any indentation, to make it easier to embed inside other formatted JSON data. Although leading space characters (space, tab, carriage return, newline) at the beginning of src are dropped, trailing space characters at the end of src are preserved and copied to dst. For example, if src has no trailing spaces, neither will dst; if src ends in a trailing newline, so will dst.
func MarshalFast ¶
func MarshalIndentFlags ¶
func MarshalPrecache ¶
func MarshalPrecacheFor ¶
func MarshalPretty ¶
func MarshalPrettyFast ¶
func ResetEncodersCache ¶
func ResetEncodersCache()
func SetMarshalMaxDeep ¶
func SetMarshalMaxDeep(deep int)
func Unmarshal ¶
Unmarshal parses the JSON-encoded data and stores the result in the value pointed to by v. If v is nil or not a pointer, Unmarshal returns an [InvalidUnmarshalError].
Unmarshal uses the inverse of the encodings that Marshal uses, allocating maps, slices, and pointers as necessary, with the following additional rules:
To unmarshal JSON into a pointer, Unmarshal first handles the case of the JSON being the JSON literal null. In that case, Unmarshal sets the pointer to nil. Otherwise, Unmarshal unmarshals the JSON into the value pointed at by the pointer. If the pointer is nil, Unmarshal allocates a new value for it to point to.
To unmarshal JSON into a value implementing Unmarshaler, Unmarshal calls that value's [Unmarshaler.UnmarshalJSON] method, including when the input is a JSON null. Otherwise, if the value implements encoding.TextUnmarshaler and the input is a JSON quoted string, Unmarshal calls encoding.TextUnmarshaler.UnmarshalText with the unquoted form of the string.
To unmarshal JSON into a struct, Unmarshal matches incoming object keys to the keys used by Marshal (either the struct field name or its tag), preferring an exact match but also accepting a case-insensitive match. By default, object keys which don't have a corresponding struct field are ignored (see [Decoder.DisallowUnknownFields] for an alternative).
To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
- bool, for JSON booleans
- float64, for JSON numbers
- string, for JSON strings
- []interface{}, for JSON arrays
- map[string]interface{}, for JSON objects
- nil for JSON null
To unmarshal a JSON array into a slice, Unmarshal resets the slice length to zero and then appends each element to the slice. As a special case, to unmarshal an empty JSON array into a slice, Unmarshal replaces the slice with a new empty slice.
To unmarshal a JSON array into a Go array, Unmarshal decodes JSON array elements into corresponding Go array elements. If the Go array is smaller than the JSON array, the additional JSON array elements are discarded. If the JSON array is smaller than the Go array, the additional Go array elements are set to zero values.
To unmarshal a JSON object into a map, Unmarshal first establishes a map to use. If the map is nil, Unmarshal allocates a new map. Otherwise Unmarshal reuses the existing map, keeping existing entries. Unmarshal then stores key-value pairs from the JSON object into the map. The map's key type must either be any string type, an integer, or implement encoding.TextUnmarshaler.
If the JSON-encoded data contain a syntax error, Unmarshal returns a [SyntaxError].
If a JSON value is not appropriate for a given target type, or if a JSON number overflows the target type, Unmarshal skips that field and completes the unmarshaling as best it can. If no more serious errors are encountered, Unmarshal returns an [UnmarshalTypeError] describing the earliest such error. In any case, it's not guaranteed that all the remaining fields following the problematic one will be unmarshaled into the target object.
The JSON null value unmarshals into an interface, map, pointer, or slice by setting that Go value to nil. Because null is often used in JSON to mean “not present,” unmarshaling a JSON null into any other Go type has no effect on the value and produces no error.
When unmarshaling quoted strings, invalid UTF-8 or invalid UTF-16 surrogate pairs are not treated as an error. Instead, they are replaced by the Unicode replacement character U+FFFD.
Types ¶
type AppendMarshaler ¶
type AppendTextMarshaler ¶
type Decoder ¶
func NewDecoder ¶
NewDecoder returns a new decoder that reads from r.
The decoder introduces its own buffering and may read data from r beyond the JSON values requested.
type Encoder ¶
func NewEncoder ¶
func (*Encoder) GrowIndent ¶
func (*Encoder) SetEscapeHTML ¶
func (*Encoder) SetFastestFlags ¶
func (e *Encoder) SetFastestFlags()
func (*Encoder) SetPrettyFlags ¶
func (*Encoder) SetStandardFlags ¶
func (e *Encoder) SetStandardFlags()
type Flags ¶
type Flags uint32
possible values: EscapeHTML, OmitEmpty, NeedQuotes
const ( // start options SortMapKeys Flags = 1 << iota EscapeHTML ValidateString ValidateTextMarshaler CompactMarshaler PrettySpaces // while encoding OmitEmpty NeedQuotes // configs EncodeFastest = 0 EncodeStandard = SortMapKeys | EscapeHTML | ValidateString | ValidateTextMarshaler | CompactMarshaler )
encoder flags
type RawMessage ¶
type RawMessage = json.RawMessage
RawMessage is a raw encoded JSON value. It implements Marshaler and Unmarshaler and can be used to delay JSON decoding or precompute a JSON encoding.
type StructField ¶
type StructField struct {
Key string
KeyLen int
Offset uintptr
Encoder UnsafeEncoder
}
type TextMarshaler ¶
type TextMarshaler = encoding.TextMarshaler
type TextMarshaler interface {
MarshalText() (text []byte, err error)
}
type TextUnmarshaler ¶
type TextUnmarshaler = encoding.TextUnmarshaler
type TextUnmarshaler interface {
UnmarshalText(text []byte) error
}
type Unmarshaler ¶
type Unmarshaler = json.Unmarshaler
type Unmarshaler interface {
UnmarshalJSON([]byte) error
}
Source Files
Directories