Challenge, Medium,  on  Kubernetes

Inject a Config File Without Overwriting the Image's Existing Directory

Scenario

The prod namespace has a webapp Deployment that is stuck in CrashLoopBackOff.

The application uses nginx:stable and has a startupProbe that runs stat /etc/nginx/conf.d/default.conf to verify the config directory is intact before the container is marked healthy. If that file is missing, the probe fails on every attempt until Kubernetes kills and restarts the container — producing the crash loop you see.

A ConfigMap named webapp-config exists in the prod namespace. It holds a single key, healthz.conf, which configures the /healthz endpoint on port 8080. The deployment already mounts this ConfigMap as a volume, but the mount is configured incorrectly — it targets the entire /etc/nginx/conf.d/ directory instead of injecting a single file into it.

When a ConfigMap is mounted at a directory path, Kubernetes replaces that entire directory with the ConfigMap contents. This removes the image's original default.conf. With default.conf gone, the startupProbe finds nothing to stat, exhausts its failure threshold, and Kubernetes restarts the container.

Investigate the deployment and the pod events to understand why the probe is failing, then fix the volume mount so the existing directory contents are preserved and healthz.conf is injected alongside them.


Task

Fix the webapp Deployment in the prod namespace so that:

  • The pod reaches Running state with 1/1 Ready
  • The /healthz endpoint on port 8080 returns HTTP 200
  • The existing /etc/nginx/conf.d/ directory content is preserved — not replaced

Do not modify the webapp-config ConfigMap, the container image, the probes, or the ports.

Fix only the volume mount configuration in the Deployment.


Hint 1 — Investigate the Crash

Inspect the pod events and the current deployment spec:

kubectl describe pod -n prod -l app=webapp
kubectl get deployment webapp -n prod -o yaml

Focus on spec.template.spec.containers[].volumeMounts and spec.template.spec.volumes.

When a ConfigMap is mounted at a directory path, Kubernetes replaces that entire directory with the ConfigMap contents. Any files that existed in the image at that path are removed — including default.conf. The startupProbe runs stat /etc/nginx/conf.d/default.conf every 5 seconds. With default.conf gone, every check fails. After 3 consecutive failures (15 seconds total), Kubernetes kills the container and restarts it.

Documentation

Hint 2 — Fix the Volume Mount

Kubernetes provides a way to mount a single key from a ConfigMap as a specific file at an exact path inside the container — without touching any other files in the target directory. This requires two changes to the volumeMount:

  • mountPath must be the full file path, not the directory
  • A second field scopes the mount to a specific key within the volume

To inject only healthz.conf into /etc/nginx/conf.d/ without replacing the directory, update the volumeMount:

volumeMounts:
  - name: config
    mountPath: /etc/nginx/conf.d/______.conf
    subPath: ______.conf

Edit the deployment and apply the change:

kubectl edit deployment webapp -n prod

Documentation

Hint 3 — Confirm the Fix

Wait for the rollout to complete, then check that the pod is running and the health endpoint responds:

kubectl rollout status deployment/webapp -n prod
kubectl exec -n prod deploy/webapp -- curl -s http://localhost:8080/healthz

If the pod is still crashing, check the pod events — they will show whether the startupProbe is still failing and give you a clue about what remains broken.


⚒ Test Cases