Recover Pending Pods
Scenario
The prod namespace has a 3d-robot Deployment with 3 replicas. The
Deployment reads its configuration from the app-config ConfigMap and uses a
nodeSelector to pin all pods to node-02. A NodePort Service exposes it on
port 30080.
All 3 pods are stuck in Pending and the Service returns no response.
Task
Find why the pods cannot be scheduled and fix the underlying issue so all 3
replicas run on node-02 and the NodePort Service is reachable on port 30080.
Do not modify the Deployment, ConfigMap, or Service.
Hint 1 — Why is the pod not starting?
A pod in Pending has been accepted by the API server but not yet placed on
any node. Start by reading what the scheduler recorded:
kubectl get pods -n prod
kubectl describe pod -n prod -l app=3d-robot
Read the Events section at the bottom carefully. It will tell you what constraint the scheduler evaluated and why no node matched.
Documentation
Hint 2 — Check the state of the cluster nodes
The scheduler only places pods on nodes that are Ready. List the nodes and
look at their status:
kubectl get nodes
kubectl describe node node-02
Check the Conditions section. A Ready status of False or Unknown
means the node is not accepting workloads.
Documentation
Hint 3 — Look at what runs on the node itself
A node goes NotReady when the control plane stops receiving reports from the
node agent. SSH to node-02 and check whether that service is active:
ssh node-02
systemctl status kubelet
If it is not running, start it and watch the node recover:
systemctl start kubelet
Then from cplane-01 confirm the node and pods recover:
kubectl get nodes --watch
kubectl get pods -n prod --watch
Documentation