re2go

func IsByline ¶

func IsByline(input string) bool

Original pattern: (?i)byline|author|dateline|writtenby|p-author

func IsNegativeClass ¶

func IsNegativeClass(input string) bool

For IsNegativeClass, its original pattern is like this: (?i)-ad-|hidden|^hid$| hid$| hid |^hid |banner|combx|comment|com-|contact|footer|gdpr|masthead|media|meta|outbrain|promo|related|scroll|share|shoutbox|sidebar|skyscraper|sponsor|shopping|tags|widget

Unfortunately, re2go doesn't handle anchor like ^ and $ internally, so for convenience I'll split that pattern into two: - `^hid$| hid$| hid |^hid ` - `-ad-|hidden|banner|combx|comment|com-|contact|footer|gdpr|masthead|media|meta|outbrain|promo|related|scroll|share|shoutbox|sidebar|skyscraper|sponsor|shopping|tags|widget`

func IsPositiveClass ¶

func IsPositiveClass(input string) bool

Original pattern: (?i)article|body|content|entry|hentry|h-entry|main|page|pagination|post|text|blog|story

func IsUnlikelyCandidates ¶

func IsUnlikelyCandidates(input string) bool

Original pattern: (?i)-ad-|ai2html|banner|breadcrumbs|combx|comment|community|cover-wrap|disqus|extra|footer|gdpr|header|legends|menu|related|remark|replies|rss|shoutbox|sidebar|skyscraper|social|sponsor|supplemental|ad-break|agegate|pagination|pager|popup|yom-remote

func MaybeItsACandidate ¶

func MaybeItsACandidate(input string) bool

Original pattern: (?i)and|article|body|column|content|main|mathjax|shadow

func NormalizeSpaces ¶

func NormalizeSpaces(input string) string

Original pattern: (?i)\s{2,}