http-mon

Table of Contents

1. HTTPs endpoint monitor
2. Sample configuration:
   1. toml format
   2. json format
   3. configuration breakdown
      1. minijq
   4. Usage:
3. Building the application
   1. Application structure
4. LICENSE

HTTPs endpoint monitor

Gathering metrics and testing custom REST apis can be challenging, this project aims to provide a simple way to monitor and gather metrics from custom REST endpoints like ElasticSearch/OpenSearch queries and REST apis.

This project implements a simple configuration DSL written in well-formed on toml or json with a simple structure. The reqeusted endpoints can fetch a number of metrics for each defined filter.

Metrics’ jq filters will get numbers from the request and output standard metrics for the resulting lists of numbers (float64 following JSON standards)

Sample configuration:

toml format

[statsd]
prefix = "monitor."
host = "localhost"
port = "8125"

[[job]]
name = "elasticsearch"
[job.request]
url = "$ES_URL/_stats/indexing,docs,merge,search"
[job.request.aws]
enable = true
service = "es"

[[job.query]]
name = "docs.primaries"
jq = "_all.primaries.docs.count"

[[job.query]]
name = "indices.documents"
jq = "indices.*.primaries.docs.count"

[[job]]
name = "oauth-service"
[job.request]
method = "post"
url = "$SERVICE_URL/products/search"
body = "{\"query\":\"cheese\"}"
[job.request.oauth]
enable = true
url = "$OAUTH_URL"
username = "$OAUTH_USERNAME"
password = "$OAUTH_PASSWORD"
jq = "token"
[[job.queries]]
name = "cheese"
jq = "items.*.price"

json format

{
    "statsd": {
        "prefix": "monitor.",
        "host": "localhost",
        "port": "8125"
    },
    "jobs": [
        {
            "name":"elasticsearch",
            "request": {
                "url": "$ES_URL/_stats/indexing,docs,merge,search",
                "aws": {
                    "enable": true,
                    "service": "es"
                }
            },
            "queries": [
                { "name": "docs.primaries", "jq": "_all.primaries.docs.count" },
                { "name": "indices.documents", "jq": "indices.*.primaries.docs.count" }
            ]
        },
        {
            "name": "oauth-service",
            "request": {
                "oauth": {
                    "enable": true,
                    "url": "$OAUTH_URL",
                    "username":"$OAUTH_USERNAME",
                    "password":"$OAUTH_PASSWORD",
                    "jq": "token"
                },
                "url": "$SERVICE_URL/products/search",
                "method": "post",
                "body": "{\"query\":\"cheese\"}"
            },
            "queries": [
                { "name": "cheese", "jq": "items.*.price" }
            ]
        }
    ]
}

configuration breakdown

• statsd the statsd server to use, defaults to localhost with port 8125, the prefix is the string prepended to the statistic name. All properties expand environment variables.
• jobs a list of jobs to perform
• jobs.name the base-name of the resulting stat.
• jobs.request HTTP/s request configuration:
  1. url: the unified resource location, eg. https://example.com/path/name
  2. method: one of the required HTTP methods, eg. GET, POST …
  3. body: string representing the body of the request
• jobs.request.aws enables AWS4 Signing for the request.
  1. enable boolean, enables AWS4 Signing
  2. service the service name to use for the AWS4 Signing procedure.
• jobs.request.oauth enables fetching an OAuth token based on username and password
  1. enable boolean, enables OAuth authorization workflow
  2. url url of the OAuth service
  3. username username for the authentication service
  4. password password for the authentication service
  5. jq the path, in the resulting JSON, where the bearer token can be found
• jobs.request.basic basic auth config
  1. enable enables basic auth
  2. username basic username
  3. password basic password
• jobs.queries
  1. name
  2. jq

minijq

Filters use a minimal jq-like query engine, it will traverse arrays and maps. It does have support for wildcards.

Examples:

{
    mapList: [
        { "prop": 1 },
        { "prop": 2 },
        { "prop": 3 },
        { "prop": 4 },
        { "prop": 5 },
    ],
    floatList: [ 1,2,3,4,5 ],
}

• mapList.0.prop will return 1.0
• mapList.*.prop will return a list of all numbers addressed by prop eg. [1.0, 2.0, 3.0, 4.0, 5.0]
• floatList.2 will return 3
• floatList returns the matching property in full eg. [1.0, 2.0, 3.0, 4.0, 5.0]
• wildcards can be nested

For more exaples check minijq_test.go.

NOTE: all numbers in JSON are parsed as float64

Usage:

monitor version 0.4.0

Usage of ./monitor:
  -c, --config string     toml config file (default "monitor.toml")
  -y, --dry               dry-run the given configuration.
  -d, --dump              dump toml config file
  -h, --help              print this help
  -T, --timing duration   time to wait between requests (default 20s)
  -v, --verbose           enable debug output

The monitor command can check the configuration using the --dump flag: ./monitor -c ./resources/configuration.toml --dump. This will not run the command and will, instead, dump the configuration in toml format.

The --dry flag executes the passed configuration requests only once.

Building the application

Go toolchain must be installed and configured. To produce a binary run: go build ./cmd/monitor or use the project’s makefile: make monitor

Application structure

cmd/monitor is the main application entrypoint, the internal folder hosts private sub-modules.

sub modules:

• config parser of the configuration, supports toml with fallback json
• core job running logic
• fetch requests parsing and authorization logic
• minijq a simple jq-like filter for interface{}
• statsd wrapped statsd client

LICENSE

Consider this program and all enclosed source files released under AGPLv2