Documentation
¶
Index ¶
- Constants
- Variables
- func Call(funcName string, stub_storage StubMapping, params ...interface{}) (result interface{}, err error)
- func Check_float64_to_int_overflow(a float64) bool
- func Euclidean_distance(i int, a []float64, b []float64) float64
- func First_words(value string, count int) string
- func GetOutboundIP() net.IP
- func Get_actual_begin(load_ptr *[]byte, separate_entries byte) (int64, error)
- func Get_actual_end(load_ptr *[]byte, separate_entries byte, offset int64) (int64, error)
- func Get_file_size(url string) int64
- func Http_download(resource string, begin int64, end int64) *[]byte
- func Join_algorithm_clustering(a interface{}, b interface{}, _ interface{}) (ret struct{})
- func MinOf_int32(vars ...int32) int32
- func Parser_simple(point *[]float64, buffered_read *bufio.Reader, separate_properties byte, ...) (full_s string, err error)
- func Rpc_job_goroutine(server *Server, load *Job, method string, log_message string, retry int, ...) interface{}
- func Rpc_request_goroutine(server *Server, load *Request, method string, log_message string, retry int, ...) interface{}
- func Welford_one_pass(mean []float64, sample []float64, nsamples float64) []float64
- type Job
- type Request
- type Server
- type StubMapping
Constants ¶
View Source
const ( EXIT_SUCCESS = 0 EXIT_FAILURE = 1 )
tech debt
View Source
const ( MASTER = "MASTER" MAPPER = "MAPPER" REDUCER = "REDUCER" )
View Source
const ( IDLE = "IDLE" BUSY = "BUSY" WAIT = "WAIT" )
View Source
const APP_ID = "SDCC"
View Source
const BYTE_SIZE = 8
View Source
const EXPIRE_TIME = 10
View Source
const MASTER_IP = "172.18.0.254"
View Source
const MAX_MAP_TASKS = 2
View Source
const SECOND = time.Second
Variables ¶
View Source
var ( WarningLoggerPtr *log.Logger InfoLoggerPtr *log.Logger ErrorLoggerPtr *log.Logger )
View Source
var MaxInt int64
Functions ¶
func Call ¶
func Call(funcName string, stub_storage StubMapping, params ...interface{}) (result interface{}, err error)
Function for calling a function by its name
func First_words ¶
Get first n (count) words of a string)
func Get_actual_begin ¶
Searching the first entry
func Get_actual_end ¶
Searching the last entry, that is the first entry after the offset (length of the slice)
func Get_file_size ¶
func Http_download ¶
Download a slice of an http resource
func Join_algorithm_clustering ¶
func Join_algorithm_clustering(a interface{}, b interface{}, _ interface{}) (ret struct{})
custom function for combining keys values
func MinOf_int32 ¶
This version of go doesn't support generics
func Parser_simple ¶
func Parser_simple(point *[]float64, buffered_read *bufio.Reader, separate_properties byte, separate_entries byte) (full_s string, err error)
Just simple parser forr comma separated values kind of files
func Rpc_job_goroutine ¶
func Rpc_job_goroutine(server *Server, load *Job, method string, log_message string, retry int, delay time.Duration, error_is_fatal bool) interface{}
connect to the rpc server and call the remote method "method" and paremeters "load" (type Job)
Types ¶
type Job ¶
type Job struct {
Id string // job id
Task_id string // task id
Origin_task_id string // task id of the first iteration
Server_id string // server id
Resource_link string // it is supposed that we are using an http resource link
Begin int64 // start of the assigned slice
End int64 // end of the assigned slice
Margin int8 // mapper doesn't download exactly its assigned slice but a bit more, it is explained in the mapper source code
Separate_entries byte // (parser) character placed between entries
Separate_properties byte // (parser) character placed between properties
Properties_amount int8 // amount of properties for each entries (eg. each cartesian plane's point has 2 properties)
Algorithm map[string]string // various algorithm that can be helpful to the job
Algorithm_parameters map[string]interface{} // same but for prameters
Result map[string]interface{}
Keys []string
Keys_x_servers map[string]map[string]*Server // amount of properties for each entries (eg. each cartesian plane's point has 2 properties)
Keys_x_servers_version int8 // if a mapper crashes then probably we have to modify the previous variable and then increment this
Delete bool
}
type StubMapping ¶
type StubMapping map[string]interface{}
Click to show internal directories.
Click to hide internal directories.