goweb

package module
v1.0.0-alpha.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 6, 2016 License: MIT Imports: 20 Imported by: 0

README

goweb

Web I/O package in pure golang.

Wraps net/http into a higer level class that handles the following:

  • Redirects
  • Cookies
  • Relative URLs
  • JSON structures
  • DOM parsing

Backwards compatibility to Golang 1.4 is currently being maintained.

Note: This project is currently unstable while the API is refined.

Dependencies: This package depends on golog

Documentation

Index

Constants

View Source 
const (
	HTTP_GET  = "GET"
	HTTP_POST = "POST"
)
View Source 
const (
	CONTENT_TYPE_DEFAULT    = ""
	CONTENT_TYPE_FORM       = "application/x-www-form-urlencoded"
	CONTENT_TYPE_JSON       = "application/json"
	CONTENT_TYPE_FORM_MULTI = "multipart/form-data"
)
View Source 
const (
	JSONArrayType = iota
	JSONDictionaryType
	JSONUnknownType
)

Variables

View Source 
var HTTP_USER_AGENT = []string{
	"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50",
	"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.75.14 (KHTML, like Gecko) Version/7.0.3 Safari/7046A194A",
	"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36",
	"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36",
}

Functions

func DetectProxy 

func DetectProxy() (result bool)

Assumes a local HTTP proxy is anything listening locally on port 8080

func EvaluateEquation 

func EvaluateEquation(javascript string) (result int, err error)

func GetClient 

func GetClient(r *http.Request) (client *http.Client)

GetClient takes an unused http.Request so we can support GAE

func IdentifityJSONFragment 

func IdentifityJSONFragment(jsonString string) (result int, index int)

IdentifityJSONFragment

func IsolateJSON 

func IsolateJSON(jsonString string, jsonTypeIn int) (result string, jsonType int)

IsolateJSON isolates the JSON parsable text based on array or dictionary delimiters

func TidyJSON 

func TidyJSON(jsonString string, jsonType int) (result string)

TidyJSON

func TidyScript 

func TidyScript(jsonString string) (result string)

TidyScript

func TidyValues 

func TidyValues(jsonString string, jsonType int) (result string)

TidyValues

func ToJSON 

func ToJSON(jsonMap JSONMapType) (result string, err error)

ToJSON

Types

type DOM 

type DOM struct {
	// contains filtered or unexported fields
}

DOM Document.

func NewDOM 

func NewDOM() *DOM

NewDOM Constructor

func (*DOM) ChildFind 

func (id *DOM) ChildFind(parent *DOMNode, tag string, attributes DOMNodeAttributes) (result []*DOMNode)

ChildFind : Find the child Node of type tag with the specified attributes

func (*DOM) ChildFindJSONForScriptWithKeyDelimiter 

func (id *DOM) ChildFindJSONForScriptWithKeyDelimiter(parent *DOMNode, substring string, jsonType int) (result JSONMapType, err error)

ChildFindJSONForScriptWithKey : Find the child JSON key with text containing substring

func (*DOM) ChildFindTextForClass 

func (id *DOM) ChildFindTextForClass(parent *DOMNode, tag string, class string) (result string)

ChildFindTextForClass : Find the child given tag with the specified attributes

func (*DOM) ChildFindWithKey 

func (id *DOM) ChildFindWithKey(parent *DOMNode, tag string, substring string) (result []*DOMNode)

ChildFindWithKey : Find the child Node of type tag with text containing key

func (*DOM) ContentLength 

func (id *DOM) ContentLength() int

ContentLength : The byte count of the raw html contents.

func (*DOM) Contents 

func (id *DOM) Contents() string

Contents : The raw html contents.

func (*DOM) Dump 

func (id *DOM) Dump()

Dump : dump the textual representation of the DOM

func (*DOM) Find 

func (id *DOM) Find(tag string, attributes DOMNodeAttributes) (result []*DOMNode)

Find : Find the Node of type tag with the specified attributes

func (*DOM) FindJSONForScriptWithKey 

func (id *DOM) FindJSONForScriptWithKey(substring string) (result JSONMapType, err error)

FindJSONForScriptWithKey : Find the JSON key with text containing substring

func (*DOM) FindJSONForScriptWithKeyDelimiter 

func (id *DOM) FindJSONForScriptWithKeyDelimiter(substring string, jsonType int) (result JSONMapType, err error)

func (*DOM) FindTextForClass 

func (id *DOM) FindTextForClass(tag string, class string) (result string)

FindTextForClass : Find the given tag with the specified attributes

func (*DOM) FindWithKey 

func (id *DOM) FindWithKey(tag string, substring string) (result []*DOMNode)

FindWithKey : Find the Node of type tag with text containing key

func (*DOM) IsChildNode 

func (id *DOM) IsChildNode(parent *DOMNode, node *DOMNode) (result bool)

IsChildNode : Is node a child of parent? The fastest confirmation is bottom up since the relationships are one to many.

func (*DOM) IsDescendantNode 

func (id *DOM) IsDescendantNode(parent *DOMNode, node *DOMNode) (result bool)

IsDescendantNode : Is node a descendant of parent? The fastest confirmation is bottom up since the relationships are one to many.

func (*DOM) RootNode 

func (id *DOM) RootNode() (result *DOMNode)

RootNode : The HTML root node

func (*DOM) SetContents 

func (id *DOM) SetContents(htmlString string)

SetContents : parse the raw html contents.

func (*DOM) String 

func (id *DOM) String() (result string)

DOM: String representation.

type DOMNode 

type DOMNode struct {
	Index         int
	Tag           string
	Attributes    DOMNodeAttributes
	TextFragments []string
	Parent        *DOMNode
	Children      []*DOMNode
}

DOMNode def

func NewDOMNode 

func NewDOMNode(index int, parent *DOMNode, tag string, attributes DOMNodeAttributes) *DOMNode

NewDOMNode constructor

func (*DOMNode) Attr 

func (id *DOMNode) Attr(key string) string

Attr Node: String with value of the provided attribute key.

func (*DOMNode) ReaderText 

func (id *DOMNode) ReaderText() (result string)

ReaderText recombines the node text fragments into the human reader visibile text

func (*DOMNode) String 

func (id *DOMNode) String() (desc string)

Node: String representation.

func (*DOMNode) Text 

func (id *DOMNode) Text() (result string)

type DOMNodeAttributes 

type DOMNodeAttributes map[string]string

DOMNodeAttributes map of strings keyed by strings

type HTML 

type HTML struct {
}

func NewHTML 

func NewHTML() *HTML

Constructor

func (*HTML) ParseRedirect 

func (self *HTML) ParseRedirect(d *DOM) (result string)

type HTTP 

type HTTP struct {
	Method   string
	URL      *url.URL
	ProxyURL *url.URL

	RawContents []byte
	// contains filtered or unexported fields
}

func NewHTTP 

func NewHTTP() *HTTP

Constructor

func (*HTTP) ContentType 

func (id *HTTP) ContentType() string

Header: Extract Content-Type

func (*HTTP) Contents 

func (id *HTTP) Contents() string

func (*HTTP) Get 

func (id *HTTP) Get(urlString string) (result string)

Fetch: GET request

func (*HTTP) GetQuery 

func (id *HTTP) GetQuery(urlString string, args map[string]string) (result string)

func (*HTTP) Host 

func (id *HTTP) Host() string

Header: Extract Hosts

func (*HTTP) JSON 

func (id *HTTP) JSON() (result map[string]interface{}, err error)

Contents: Marshall contents from JSON to a map if possible

func (*HTTP) Location 

func (id *HTTP) Location() string

Header: Extract Location

func (*HTTP) Post 

func (id *HTTP) Post(urlString string, args map[string]string) (result string)

Fetch: POST request

func (*HTTP) PostContent 

func (id *HTTP) PostContent(urlString string, contentType string, content *bytes.Buffer) (result string)

Fetch: POST request

func (*HTTP) PostData 

func (id *HTTP) PostData(urlString string, contentType string, contentBytes []byte) (result string)

func (*HTTP) PostString 

func (id *HTTP) PostString(urlString string, contentType string, contentString string) (result string)

func (*HTTP) Referer 

func (id *HTTP) Referer() string

Header: Extract Referer

func (*HTTP) SetGAERequest 

func (id *HTTP) SetGAERequest(req *http.Request)

func (*HTTP) Status 

func (id *HTTP) Status() int

Header: Extract Status

func (*HTTP) URLString 

func (id *HTTP) URLString() (urlString string)

type JSONMapType 

type JSONMapType map[string]interface{}

func ExtractJSON 

func ExtractJSON(jsonString string, jsonType int) (result JSONMapType, err error)

ExtractJSON isolates and tidys JSON string while attepting to parse the contents into a JSONMap

func FromJSON 

func FromJSON(jsonString string) (result JSONMapType, err error)

FromJSON

type JSONSliceType 

type JSONSliceType []interface{}

type JScript 

type JScript struct {
}

func NewJScript 

func NewJScript() *JScript

Constructor

func (*JScript) ParseRedirect 

func (self *JScript) ParseRedirect(d *DOM) string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.