timeshard

package module
v0.0.0-...-c26f1b5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 7, 2019 License: GPL-3.0 Imports: 13 Imported by: 0

README

TimeShard

TimeShard is an open-source operational transform time series database. It was primarily designed, to store OT-Operations from web edited documents (by sending only the modified bits).

Documentation Go report card License

Currently implemented

  • Insert Operation with Retain to move the cursor.
  • Delete Operation with Retain
  • Data snapshots
  • Squash (See how a document looked in a point in time)
  • JSON Export
  • Error recovery
  • Disk persistence with Snappy
  • Text Formatting

Inserting data

batch := timeshard.NewBatch()

text := []byte("this is a long text")
batch.Insert(0, text)

Deleting data

batch.Delete(0, 10)

Iterators

snap := batch.Snapshot()

iter := snap.Iterator(true)
for iter.HasNext() {
	//iter.Value()
}

FAQ

Q: Is it safe to iterate while adding new data?

A: You can only iterate over snapshots (a copy of a document, at a certain period of time). Snapshots cannot be edited, but you can create Batches from them.

Q: Is it production ready?

A: No. Only if you treat bugs as features.

Documentation

Index

Constants

View Source 
const (
	// Will Insert N runes at position
	OpInsert = 0

	// Will Delete N runes at position
	OpDelete = 1
)
View Source 
const MetaSize = 5

The size of our Meta

Variables

This section is empty.

Functions

This section is empty.

Types

type Block 

type Block struct {
	Shard
	sync.Mutex
}

func Merge 

func Merge(src *Block, dst *Block) (*Block, error)

Will merge the instructions from two Snapshots

func NewBlock 

func NewBlock() *Block

func (*Block) Clone 

func (block *Block) Clone() *Block

Clone will clone the current snapshot, useful for iterating

func (*Block) Delete 

func (block *Block) Delete(at uint64, count uint64)

Delete will add an OpDelete into our Shard TODO: Use add

func (*Block) Insert 

func (block *Block) Insert(at uint64, rawBytes []byte)

Insert will add an OpInsert into our Shard

func (*Block) Iterator 

func (block *Block) Iterator(reverse bool) Iterator

func (*Block) LastActivity 

func (block *Block) LastActivity() uint64

func (*Block) MarshalJSON 

func (block *Block) MarshalJSON() ([]byte, error)

func (*Block) Squash 

func (block *Block) Squash(count uint64) *Block

func (*Block) UnmarshalJSON 

func (block *Block) UnmarshalJSON(data []byte) error

type Document 

type Document struct {
	Title      string            `json:"title"`
	Operations Block             `json:"ops"`
	Meta       map[string]string `json:"meta"`
}

func NewDocument 

func NewDocument() Document

func (*Document) Bytes 

func (doc *Document) Bytes() []byte

Bytes returns a slice of length b.Len() holding the end result of our operations

func (*Document) Delete 

func (doc *Document) Delete(at uint64, count uint64) *Document

func (*Document) Each 

func (doc *Document) Each(evalFunc func(current Iterator) bool) *Document

func (*Document) FromBytes 

func (doc *Document) FromBytes(compressed []byte) (err error)

func (*Document) Insert 

func (doc *Document) Insert(at uint64, rawBytes []byte) *Document

func (*Document) Open 

func (doc *Document) Open(filename string) (err error)

func (*Document) Save 

func (doc *Document) Save(filename string) (err error)

func (*Document) String 

func (doc *Document) String() string

String returns the contents of the document as a string

func (*Document) Write 

func (doc *Document) Write(file io.Writer) (int, error)

type ForwardIterator 

type ForwardIterator struct {
	// contains filtered or unexported fields
}

func (*ForwardIterator) GetMeta 

func (it *ForwardIterator) GetMeta(metaType MetaType) uint64

func (*ForwardIterator) HasNext 

func (it *ForwardIterator) HasNext() bool

func (*ForwardIterator) Init 

func (it *ForwardIterator) Init()

func (*ForwardIterator) Meta 

func (it *ForwardIterator) Meta() []uint64

func (*ForwardIterator) Value 

func (it *ForwardIterator) Value() []byte

type Iterator 

type Iterator interface {
	Init()

	HasNext() bool

	Meta() []uint64
	GetMeta(MetaType) uint64
	Value() []byte
}

type JSONOperation 

type JSONOperation struct {
	Position uint64 `json:"p"`
	Insert   string `json:"insert,omitempty"`
	Delete   uint64 `json:"delete,omitempty"`
}

type MetaType 

type MetaType uint64
const (
	// Where data is stored in our slice, index of []byte
	MetaDataIndex MetaType = iota

	// How many runes this action will affect (i.e. for DeleteOp, how many runes we will delete)
	// For InsertOp, the size of our data in bytes (used for keeping data in the slice)
	MetaDataByteSize

	// The type of our operation Delete or Insert
	MetaOperation

	// How many runes to skip
	MetaRetain

	// When was the action triggered, used for sorting and keeping data integrity
	MetaTimestamp
)

type ReverseIterator 

type ReverseIterator struct {
	// contains filtered or unexported fields
}

func (*ReverseIterator) AffectedArea 

func (it *ReverseIterator) AffectedArea() uint64

func (*ReverseIterator) GetMeta 

func (it *ReverseIterator) GetMeta(metaType MetaType) uint64

func (*ReverseIterator) HasNext 

func (it *ReverseIterator) HasNext() bool

func (*ReverseIterator) Init 

func (it *ReverseIterator) Init()

func (*ReverseIterator) Meta 

func (it *ReverseIterator) Meta() []uint64

func (*ReverseIterator) PointInTime 

func (it *ReverseIterator) PointInTime() uint64

func (*ReverseIterator) Retain 

func (it *ReverseIterator) Retain() uint64

func (*ReverseIterator) Type 

func (it *ReverseIterator) Type() uint64

func (*ReverseIterator) Value 

func (it *ReverseIterator) Value() []byte

type Shard 

type Shard struct {
	// contains filtered or unexported fields
}

func NewShard 

func NewShard() Shard

func (*Shard) Get 

func (s *Shard) Get(index int, k MetaType) (uint64, bool)

Get returns two things, the meta value and a boolean if the value was found. If the index overflows our slice, it will return 0 and false

func (*Shard) Init 

func (s *Shard) Init()

func (*Shard) IsEmpty 

func (s *Shard) IsEmpty() bool

IsEmpty asserts whether we have 0 actions or not in our shard. I.E. An empty Batch will return 0

func (Shard) Len 

func (s Shard) Len() int

Len returns the length of a shard (how many operations it produces) We get this by dividing the length of our meta slice, to the MetaSize count constant I.E. A batch with one insert will return 1 when we call Len()

func (Shard) Less 

func (s Shard) Less(i, j int) bool

func (*Shard) Pop 

func (s *Shard) Pop() interface{}

func (*Shard) Push 

func (s *Shard) Push(x interface{})

func (Shard) Swap 

func (s Shard) Swap(i, j int)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.