Documentation
¶
Overview ¶
Package mapreduce implements the Map/Reduce algorithm for the processing and aggregation mass data.
A type implementing the MapReducer interface has to be implemented and passed to the MapReduce() function. The type is responsible for the input, the mapping, the reducing and the consuming while the package provides the runtime environment for it.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func MapReduce ¶
func MapReduce(mr MapReducer) error
MapReduce applies a map and a reduce function to keys and values in parallel.
Types ¶
type Identifiable ¶
type Identifiable interface {
// ID returns the identifier for the mapping.
ID() string
}
Identifiable has to be implemented by the data handled by map/reduce.
type IdentifiableChan ¶
type IdentifiableChan chan Identifiable
IdentifiableChan is a channel for the transfer of identifiable data.
func (IdentifiableChan) Close ¶
func (c IdentifiableChan) Close()
Close the channel for identifiable data.
type MapReducer ¶
type MapReducer interface {
// Input has to return the input channel for the
// date to process.
Input() IdentifiableChan
// Map maps a key/value pair to another one and emits it.
Map(in Identifiable, emit IdentifiableChan)
// Reduce reduces the values delivered via the input
// channel to the emit channel.
Reduce(in, emit IdentifiableChan)
// Consume allows the MapReducer to consume the
// processed data.
Consume(in IdentifiableChan) error
}
MapReducer has to be implemented to control the map/reducing.
Source Files
Click to show internal directories.
Click to hide internal directories.