bigip-ctlr/

= What's this
:toc: manual

This project facilitates quick debugging of CIS by setting up Kubernetes, VE, and CIS externally from Kubernetes. It allows running the source code directly and provides a friendly debugging environment.

== CIS 2.9.1

[source, bash]
.*1. Use kubekey to create a K8S Cluster*
----
sudo ./kk create cluster
----

More details above kubekey refer to https://github.com/kubesphere/kubekey.

Once K8S Cluster created, copy `.kube/config` to source code folder.

[source, bash]
.*2. Start Mock AS3*
----
cd ../as3-mock
go mod init github.com/kylinsoong/golang/as3-mock
go mod tidy
go run cmd/as3-mock/main.go --tls-certificate=$(pwd)/cert/certificate.crt --tls-private=$(pwd)/cert/private.key
----

[source, bash]
.*3. Start CIS*
----
go mod init github.com/kylinsoong/bigip-ctlr
go mod tidy
go run cmd/bigip-ctlr/*.go --bigip-username=admin --bigip-password=admin --bigip-url=192.168.45.52 --log-level=DEBUG --running-in-cluster=false --kubeconfig=/Users/k.song/src/golang/bigip-ctlr/config --namespace=f5-hub-1 --namespace=f5-hub-2 --node-poll-interval=300 --periodic-sync-interval=36000 --hubmode=true --manage-ingress=false --manage-configmaps=true --insecure=true --filter-tenants=true --pool-member-type=cluster --bigip-partition=k8s --log-as3-response=true --schema-db-base-dir=file:///Users/k.song/src/golang/bigip-ctlr/schemas/

go run cmd/bigip-ctlr/*.go --bigip-username=admin --bigip-password=admin --bigip-url=127.0.0.1 --log-level=DEBUG --running-in-cluster=false --kubeconfig=/Users/k.song/src/golang/bigip-ctlr/config --namespace=f5-hub-1 --namespace=f5-hub-2 --node-poll-interval=300 --periodic-sync-interval=36000 --hubmode=true --manage-ingress=false --manage-configmaps=true --insecure=true --filter-tenants=true --pool-member-type=cluster --bigip-partition=k8s --log-as3-response=true --schema-db-base-dir=file:///Users/k.song/src/golang/bigip-ctlr/schemas/
----

[source, bash]
.*4. Start Deploy APP*
----
git clone https://github.com/cloudadc/cis-scripts.git -b cis-hub
cd cis-scripts

kubectl apply -f 001/app.yaml
kubectl apply -f 002/app.yaml
kubectl apply -f 003/app.yaml

kubectl apply -f 003/cm.yaml
----

[source, bash]
.*5. Make API server temporary unavailable*
----
iptables -A INPUT -p tcp --dport 6443 -j REJECT --reject-with tcp-reset
----

add API server back via

[source, bash]
----
iptables -D INPUT -p tcp --dport 6443 -j REJECT --reject-with tcp-reset
----

[source, bash]
.*6. Review the CIS log, empty member post body be sent to VE*
----
2024/04/19 13:06:12 [DEBUG] [AS3] Posting AS3 Declaration - processFilterTenants, partition: cistest001, declaration: {"$schema":"https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/3.36.1/as3-schema-3.36.1-1.json","class":"AS3","declaration":{"cistest001":{"app-1":{"app_1_svc_pool":{"class":"Pool","loadBalancingMode":"least-connections-member","members":[{"serverAddresses":[],"servicePort":8080}],"monitors":["tcp"]},"app_svc_vs":{"class":"Service_HTTP","persistenceMethods":["cookie"],"pool":"app_1_svc_pool","snat":"self","virtualAddresses":["192.168.200.31"],"virtualPort":80},"class":"Application","template":"generic"},"class":"Tenant","defaultRouteDomain":0},"class":"ADC","controls":{"class":"Controls","userAgent":"CIS Configured AS3"},"id":"urn:uuid:85626792-9ee7-46bb-8fc8-4ba708cfdc1d","label":"CIS Declaration","remark":"Auto-generated by CIS","schemaVersion":"3.36.0"}}
2024/04/19 13:06:12 [DEBUG] [AS3] posting request to https://127.0.0.1/mgmt/shared/appsvcs/declare/cistest001
2024/04/19 13:06:12 [DEBUG] [AS3] Response from BIG-IP: code: 200, tenant: cistest001, message: success, runTime: 100
2024/04/19 13:06:12 [DEBUG] [AS3] Posting AS3 Declaration - processFilterTenants, partition: cistest002, declaration: {"$schema":"https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/3.36.1/as3-schema-3.36.1-1.json","class":"AS3","declaration":{"cistest002":{"app-1":{"app_1_svc_pool":{"class":"Pool","loadBalancingMode":"least-connections-member","members":[{"serverAddresses":[],"servicePort":8080}],"monitors":["tcp"]},"app_svc_vs":{"class":"Service_HTTP","persistenceMethods":[{"use":"cookie_encryption"}],"pool":"app_1_svc_pool","snat":"self","virtualAddresses":["192.168.200.32"],"virtualPort":80},"class":"Application","cookie_encryption":{"class":"Persist","cookieMethod":"insert","encrypt":true,"passphrase":{"ciphertext":"a3RjeGZ5Z2Q=","protected":"eyJhbGciOiJkaXIiLCJlbmMiOiJub25lIn0="},"persistenceMethod":"cookie"},"template":"generic"},"class":"Tenant","defaultRouteDomain":0},"class":"ADC","controls":{"class":"Controls","userAgent":"CIS Configured AS3"},"id":"urn:uuid:85626792-9ee7-46bb-8fc8-4ba708cfdc1d","label":"CIS Declaration","remark":"Auto-generated by CIS","schemaVersion":"3.36.0"}}
2024/04/19 13:06:12 [DEBUG] [AS3] posting request to https://127.0.0.1/mgmt/shared/appsvcs/declare/cistest002
2024/04/19 13:06:12 [DEBUG] [AS3] Response from BIG-IP: code: 200, tenant: cistest002, message: success, runTime: 100
2024/04/19 13:06:12 [DEBUG] [AS3] Posting AS3 Declaration - processFilterTenants, partition: cistest003, declaration: {"$schema":"https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/3.36.1/as3-schema-3.36.1-1.json","class":"AS3","declaration":{"cistest003":{"app-1":{"app_1_svc_pool":{"class":"Pool","loadBalancingMode":"least-connections-member","members":[{"serverAddresses":[],"servicePort":80}],"monitors":[{"use":"custom_http_monitor"}]},"app_svc_vs":{"class":"Service_HTTP","persistenceMethods":["cookie"],"pool":"app_1_svc_pool","snat":"self","virtualAddresses":["192.168.200.33"],"virtualPort":80},"class":"Application","custom_http_monitor":{"class":"Monitor","interval":5,"monitorType":"http","receive":"coffee","send":"GET /coffee/ HTTP/1.1\r\nHost:f5.monitor.com\r\n\r\n","timeout":16},"template":"generic"},"class":"Tenant","defaultRouteDomain":0},"class":"ADC","controls":{"class":"Controls","userAgent":"CIS Configured AS3"},"id":"urn:uuid:85626792-9ee7-46bb-8fc8-4ba708cfdc1d","label":"CIS Declaration","remark":"Auto-generated by CIS","schemaVersion":"3.36.0"}}
2024/04/19 13:06:12 [DEBUG] [AS3] posting request to https://127.0.0.1/mgmt/shared/appsvcs/declare/cistest003
----

Please look out to serverAddresses section:

[source, json]
----
{
    "$schema": "https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/3.36.1/as3-schema-3.36.1-1.json",
    "class": "AS3",
    "declaration": {
        "cistest003": {
            "app-1": {
                "app_1_svc_pool": {
                    "class": "Pool",
                    "loadBalancingMode": "least-connections-member",
                    "members": [
                        {
                            "serverAddresses": [],
                            "servicePort": 80
                        }
                    ],
                    "monitors": [
                        {
                            "use": "custom_http_monitor"
                        }
                    ]
                },
                "app_svc_vs": {
                    "class": "Service_HTTP",
                    "persistenceMethods": [
                        "cookie"
                    ],
                    "pool": "app_1_svc_pool",
                    "snat": "self",
                    "virtualAddresses": [
                        "192.168.200.33"
                    ],
                    "virtualPort": 80
                },
                "class": "Application",
                "custom_http_monitor": {
                    "class": "Monitor",
                    "interval": 5,
                    "monitorType": "http",
                    "receive": "coffee",
                    "send": "GET /coffee/ HTTP/1.1\r\nHost:f5.monitor.com\r\n\r\n",
                    "timeout": 16
                },
                "template": "generic"
            },
            "class": "Tenant",
            "defaultRouteDomain": 0
        },
        "class": "ADC",
        "controls": {
            "class": "Controls",
            "userAgent": "CIS Configured AS3"
        },
        "id": "urn:uuid:85626792-9ee7-46bb-8fc8-4ba708cfdc1d",
        "label": "CIS Declaration",
        "remark": "Auto-generated by CIS",
        "schemaVersion": "3.36.0"
    }
}
----

NOTE: the VS not be changed, and the pool member be deleted.

== CIS 2.16

Use same steps in *CIS 2.9.1* to create a K8S Cluster ans set up Mock As3 Service.

[source, bash]
.*1. Start CIS*
----
cd k8s-bigip-ctlr-2.16.0
go mod tidy
go run cmd/k8s-bigip-ctlr/*.go --bigip-username=admin --bigip-password=admin --bigip-url=127.0.0.1 --log-level=DEBUG --running-in-cluster=false --kubeconfig=/Users/k.song/src/golang/bigip-ctlr/config --namespace=f5-hub-1 --namespace=f5-hub-2 --node-poll-interval=300 --periodic-sync-interval=36000 --hubmode=true --manage-ingress=false --manage-configmaps=true --insecure=true --filter-tenants=true --pool-member-type=cluster --bigip-partition=k8s --log-as3-response=true --schema-db-base-dir=file:///Users/k.song/src/cis/k8s-bigip-ctlr-2.16.0-debug/schemas/
----

NOTE: Source code can be download from https://github.com/F5Networks/k8s-bigip-ctlr/archive/refs/tags/v2.16.0.zip

NOTE: `config` file should be copy from K8S Host.

NOTE: Comment out lines in `cmd/k8s-bigip-ctlr/main.go` from 1119 to 1152.

Use same steps in *CIS 2.9.1* to deploy app, configmap, and use iptables to make api server temporary unavailable.

[source, bash]
.*2. Review the CIS log and AS3 Mock service log*
----
2024/04/19 16:00:07 [ERROR] [CORE] Error getting service list. Get "https://10.1.10.137:6443/api/v1/services?labelSelector=cis.f5.com%2Fas3-tenant%3Dcistest001%2Ccis.f5.com%2Fas3-app%3Dapp-1%2Ccis.f5.com%2Fas3-pool%3Dapp_1_svc_pool": dial tcp 10.1.10.137:6443: connect: connection refused
2024/04/19 16:00:36 [ERROR] [CORE] Error getting service list. Get "https://10.1.10.137:6443/api/v1/services?labelSelector=cis.f5.com%2Fas3-tenant%3Dcistest001%2Ccis.f5.com%2Fas3-app%3Dapp-1%2Ccis.f5.com%2Fas3-pool%3Dapp_1_svc_pool": dial tcp 10.1.10.137:6443: connect: connection refused
2024/04/19 16:01:06 [ERROR] [CORE] Error getting service list. Get "https://10.1.10.137:6443/api/v1/services?labelSelector=cis.f5.com%2Fas3-tenant%3Dcistest001%2Ccis.f5.com%2Fas3-app%3Dapp-1%2Ccis.f5.com%2Fas3-pool%3Dapp_1_svc_pool": dial tcp 10.1.10.137:6443: connect: connection refused
2024/04/19 16:01:36 [ERROR] [CORE] Error getting service list. Get "https://10.1.10.137:6443/api/v1/services?labelSelector=cis.f5.com%2Fas3-tenant%3Dcistest001%2Ccis.f5.com%2Fas3-app%3Dapp-1%2Ccis.f5.com%2Fas3-pool%3Dapp_1_svc_pool": dial tcp 10.1.10.137:6443: connect: connection refused
2024/04/19 16:02:06 [ERROR] [CORE] Error getting service list. Get "https://10.1.10.137:6443/api/v1/services?labelSelector=cis.f5.com%2Fas3-tenant%3Dcistest001%2Ccis.f5.com%2Fas3-app%3Dapp-1%2Ccis.f5.com%2Fas3-pool%3Dapp_1_svc_pool": dial tcp 10.1.10.137:6443: connect: connection refused
---- 

NOTE: No empty member post payload be push to AS3. The CIS log hints errors.