Incident replay — the exposed dashboard (JW Player's cryptominer)
The real incident
JW Player found a cryptocurrency miner running on their internal Kubernetes clusters. Not a sophisticated nation-state operation — an internal tool (Weave Scope) exposed to the public internet via a cloud LoadBalancer with no authentication. Scope offers a UI to launch commands in containers. To an attacker scanning for open dashboards, that's a free "run my miner here" button against someone else's compute bill.
Source: How a cryptocurrency miner made its way onto our internal Kubernetes clusters — JW Player
The lesson is unglamorous and universal: the breach wasn't a Kubernetes
vulnerability — it was exposure plus missing auth. type: LoadBalancer /
NodePort on an internal tool is a door to the internet, and the internet scans
every door continuously.
This cluster, right now
Two findings sit in kubelings:
ops-console— an internal tool onNodePort 31337. Public. No auth.sys-helper— a Deployment in no git repo, its pods busy-looping ("hashing"), labeledworkload=xmrig-lookalike. The intruder.
kubectl -n kubelings get deploy,svc -o wide
kubectl -n kubelings get pods --show-labels
┌────────────────────┐
│public LoadBalancer │
│ │
└────────────────────┘
│
reachable by anyone
│
▼
┌──────────────────┐
│Scope UI, no auth │
│ │
└──────────────────┘
│
by design
│
▼
┌───────────────────────┐
│commands in containers │
│ │
└───────────────────────┘
Your task
Incident response, correct order:
- Contain the active threat: remove the
sys-helperminer workload. - Close the entry: stop
ops-consolebeing publicly reachable — make itClusterIP(or delete the Service). Kill the process then the door, so the attacker can't just redeploy while you're cleaning up.
Hint
# 1. evict the intruder
kubectl -n kubelings delete deploy sys-helper
# 2. shut the public door
kubectl -n kubelings patch svc ops-console --type=merge -p '{"spec":{"type":"ClusterIP"}}'
Solution
Response order matters
Real IR sequence, mirrored by this lesson:
- Contain — stop the bleeding (kill the miner). If you closed the door first but left the miner, it keeps burning compute and may re-establish.
- Close the vector — the exposed Service. Do this immediately after containment or the attacker re-enters through the same door mid-cleanup.
- (Real world, beyond this lab) Eradicate & investigate — rotate every credential the compromised pods could reach, audit for persistence (extra RBAC bindings, cron pods, mutating webhooks), preserve logs, dig for how far in they got. A miner is often the loudest thing an attacker does, not the only thing.
The defenses that would have stopped it (layers)
- Exposure hygiene: internal tools never get
LoadBalancer/NodePort. Reach them viakubectl port-forwardor an authenticated ingress. Audit:kubectl get svc -A | grep -vE 'ClusterIP|kube-system'. - Auth in front of every UI — no unauthenticated dashboard, ever, internal or not.
- RBAC least-privilege (previous lesson): even having landed, the miner's SA should be able to do almost nothing.
- Pod Security / NetworkPolicy (next lessons): block privileged pods and egress to mining pools, so the payload fails even if launched.
- Runtime detection: a pod pinning CPU with an unknown image + a workload absent from git = two alertable signals.
Why this lives in Security, not Networking
The NodePort is Module 4 mechanics — but the failure is a security-model gap: "internal" was assumed, never enforced. Every exposure decision is a security decision. The Incident Library has more of these; they nearly all reduce to a door someone forgot was a door.
- Previous lesson
- cluster-admin for a bot: scope it down