Lock Down Container Syscalls With Seccomp Profiles
Scenario
The security team is rolling out seccomp in the beacon namespace. Seccomp limits which Linux system calls (syscalls) a container can make.
| Deployment | State |
|---|---|
beacon-api | Runs with seccomp explicitly turned off (Unconfined) at the Pod level. |
beacon-worker | Has no seccomp settings. 2 replicas, one on each worker node. |
beacon-sync | Uses a custom profile, beacon-strict.json, and keeps crashing. 2 replicas, one on each worker node. |
Task
- Make the
beacon-apiDeployment in thebeaconnamespace run with the seccomp profile that the container runtime provides. - Apply the profile
/home/laborant/seccomp/no-chmod.jsonto theworkercontainer of thebeacon-workerDeployment in thebeaconnamespace. Set it in the container's ownsecurityContext. Both replicas must run with it, on whichever worker node they are scheduled. - Fix the
beacon-syncDeployment in thebeaconnamespace. Its Pods crash at startup because thebeacon-strict.jsonprofile blocks themkdircommand.- On both
node-01andnode-02, removemkdirandmkdiratfrom the profile'snameslist. Keep the other six syscalls in the list. - Restart the Deployment. It must still use
profiles/beacon-strict.json.
- On both
Work on dev-machine as the default laborant user. The beacon Pods run only on the worker nodes, node-01 and node-02, because the control plane node cplane-01 is tainted. You can reach the worker nodes from dev-machine over SSH, and laborant can use sudo on them.
Do not change the container images or commands, and do not run any container as privileged.
Hint 1: Where seccomp is set in a Pod
Seccomp is part of the securityContext, at Pod level or container level:
securityContext:
seccompProfile:
type: <type>
localhostProfile: <path> # used by one of the types only
kubectl explain pod.spec.securityContext.seccompProfile
Hint 2: Checking whether a container is filtered
The kernel reports the seccomp mode of every process:
kubectl exec <pod> -n <namespace> -- grep Seccomp /proc/1/status
0 means no filter, 2 means a filter is active.
Hint 3: Where custom profiles live
In this cluster, custom profiles are kept on each node in:
/var/lib/kubelet/seccomp/profiles/
localhostProfile is relative to the kubelet's seccomp directory, /var/lib/kubelet/seccomp/, so a file at /var/lib/kubelet/seccomp/profiles/app.json is referenced as profiles/app.json. The directory belongs to root. To look at the profiles on a node, connect to it:
ssh <node>
When the kubelet cannot load a profile, the reason is in the Pod's events:
kubectl describe pod <pod> -n <namespace>
Hint 4: Fixing beacon-strict.json
The profile is the /var/lib/kubelet/seccomp/profiles/beacon-strict.json file on each worker node. A profile is a JSON file with a defaultAction for every syscall and a list of syscalls rules that override it.
The mkdir command can use either the mkdir syscall or its newer form mkdirat. That is why both names must go.
The file belongs to root. Connect to each worker node with ssh and edit the file there with sudo and an editor such as vi.
After the edit, the file must still be valid JSON, so watch the commas between the names.
The kubelet reads a profile when a container starts, so a container that is already running keeps the filter it started with.