Lesson  in  DevSecOps in Practice: Container & Kubernetes Security

Runtime Threat Detection with Falco

Install Falco using its modern eBPF driver. Deploy a test pod, then trigger real security events — spawn a shell inside the container, read sensitive files, make outbound connections. Read the Falco alerts and understand how to route them to a SIEM.

Why Runtime Detection?

Static scanning (Trivy) finds vulnerabilities in images before they run. Falco detects attacks while they're happening — a shell spawned inside a container, a read of /etc/shadow, an unexpected outbound connection.

An image can be CVE-free and still be used in an attack. You need both.

Install Falco

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update

helm install falco falcosecurity/falco \
  --namespace falco \
  --create-namespace \
  --set driver.kind=modern_ebpf \
  --set tty=true \
  --set falco.json_output=true

echo "Waiting for Falco to initialize (2-3 minutes)..."
kubectl wait --for=condition=Ready pods --all -n falco --timeout=180s
echo "Falco is running."

Verify Falco is monitoring

kubectl get pods -n falco
kubectl logs -n falco -l app.kubernetes.io/name=falco -c falco --tail=20

Expected:

Enabled event sources: syscall
Opening 'syscall' source with modern BPF probe.
Loading rules file ...

Deploy a Test Workload

The test pod runs as root inside the container — this is the common misconfiguration attackers exploit after container escape.

kubectl create namespace falco-test

cat > test-pod.yaml << 'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: test-workload
  namespace: falco-test
spec:
  containers:
  - name: app
    image: ubuntu:22.04
    command: ["sleep", "3600"]
EOF

kubectl apply -f test-pod.yaml
kubectl wait --for=condition=Ready pod/test-workload -n falco-test --timeout=60s
echo "Test pod ready."

Trigger Security Events

Spawn a shell inside the container

Falco's "Terminal shell in container" rule fires when a shell runs with a TTY attached (-it flags). This is how an attacker would explore the container after gaining access.

kubectl exec -it -n falco-test test-workload -- bash

Inside the shell, type a few commands then exit:

id
whoami
exit

Check the Falco alert (run from your terminal, not inside the exec):

sleep 3
kubectl logs -n falco -l app.kubernetes.io/name=falco -c falco --tail=20 | grep -i "shell\|Terminal"

Expected Falco alert:

{"rule":"Terminal shell in container","priority":"Notice",
 "output":"A shell was spawned in a container ... k8s.ns=falco-test k8s.pod=test-workload"}

Why -it? The "Terminal shell in container" rule checks proc.tty != 0. Without -t, no TTY is allocated, proc.tty == 0, and the rule never fires. Always use -it when demonstrating this.

Read a sensitive file

This fires without a TTY — any open() on /etc/shadow triggers "Read sensitive file untrusted" at WARNING priority.

kubectl exec -n falco-test test-workload -- cat /etc/shadow
sleep 3
kubectl logs -n falco -l app.kubernetes.io/name=falco -c falco --tail=20 | grep -i "sensitive\|shadow"

Expected Falco alert:

{"rule":"Read sensitive file untrusted","priority":"Warning",
 "output":"Sensitive file opened for reading ... file=/etc/shadow k8s.ns=falco-test k8s.pod=test-workload"}

View all alerts

kubectl logs -n falco -l app.kubernetes.io/name=falco -c falco --tail=100 | \
  grep '"rule"' | \
  python3 -c "
import sys, json
for line in sys.stdin:
    try:
        e = json.loads(line.strip())
        p = e.get('priority','')
        r = e.get('rule','')
        prefix = {'Notice':'[Notice]','Warning':'[WARNING]','Error':'[ERROR]','Critical':'[CRITICAL]'}.get(p,'[?]')
        print(f'{prefix} {r}')
    except: pass
" 2>/dev/null | sort | uniq -c | sort -rn

Alert Routing in Production

Falco → falcosidekick → Slack / PagerDuty / Splunk

Add to your helm install:

--set falcosidekick.enabled=true \
--set falcosidekick.config.slack.webhookurl="https://hooks.slack.com/..."

Cleanup

kubectl delete namespace falco-test
helm uninstall falco -n falco
kubectl delete namespace falco
Previous lesson
Secret Scanning with Gitleaks
Start lesson