If you try to limit the memory usage of a container with a multi-process application, you might be surprised to find that when the OOM killer terminates one of the processes, the container will often keep running. Only if the terminated process is the topmost process of the container (PID 1), the container will exit.
If the containerized application cannot gracefully handle the termination of child processes caused by memory exhaustion, this behavior can be undesirable. Instead, the entire process group should be terminated, allowing the container to exit, and potentially be restarted by the orchestrator.
Under the hood, Docker relies on cgroups to manage the memory usage of containers.
Luckily, in cgroup v2, there is a setting to control exactly this behavior.
Writing 1 to the cgroup's memory.oom.group file will make the OOM killer terminate the entire process group.
However, there doesn't seem to be a corresponding flag in the docker run command yet.
Can you find a way around it?
Your task is to start a container with a multi-process application in such a way that the entire container is terminated when its total memory usage exceeds 500 MB:
- Use the
ghcr.io/iximiuz/labs/resource-hog/herder:v1.0.0image. - Set the memory limit to 500 MB with
docker run --memory 500mflag. - Name the container
herder- the solution checker expects this name.
Good luck!
Hint 1 💡
This challenge builds upon the Kill the Entire Process Group When One Process Goes Out-of-Memory challenge, so you may want to solve that first if you haven't already.
Hint 2 💡
One way to achieve this is to start the container, then find its cgroup and write 1 to its memory.oom.group file.
It's not the most elegant way to do it,
because there will be a gap between the container start and the moment its cgroup is properly adjusted.
However, it might be the only way for Docker today - even when the --cgroup-parent flag is used,
Docker seemingly sets the memory.oom.group setting of the container's and its parent cgroups to 0 during the container start.
Interesting fact: nerdctl and podman do allow fine-tuning cgroup configuration before container start with the --cgroup-conf flag:
nerdctl run --cgroup-conf=memory.oom.group=1 ...
Hint 3 💡
Don't know how to find the container's cgroup?
Try listing all server's cgroups with systemd-cgls and see if you can spot some Docker-related items in the output.
Hint 4 💡
Do not overthink this challenge.
Just starting the container with docker run --memory 500m will already create a cgroup for it.
You only need to locate the created by Docker cgroup and write 1 to its memory.oom.group file.