Kubernetes Pod Scheduling: Taints, Tolerations and Node Affinity - Why You May Need Both

Taints and tolerations repel unwanted pods from a node, but tolerated pods are not required to land on the tainted node. Node affinity attracts pods to a specific node, but nothing blocks neutral pods from landing there too. Combining both is the only way to get full control over pod placement on a dedicated node.

• Taints and Tolerations - Kubernetes Docs
• Node Affinity - Kubernetes Docs

The cluster is pre-configured with node-01 labeled node-type=dedicated. Run kubectl get nodes --show-labels to confirm before starting.

Task 1 - Taints Restrict Pod Placement

Tainting both worker nodes with NoExecute leaves only cplane-01 available to pods without a matching toleration. neutral-pod has nowhere else to go, so the scheduler places it on cplane-01.

Steps:

• Taint node-01 with key dedicated, value true, effect NoExecute
• Taint node-02 with key dedicated, value true, effect NoExecute
• Create a pod named neutral-pod, image nginx:alpine, no toleration

Task 2 - Pods with a Toleration May Land Anywhere

A toleration is permission, not a requirement. Pods with a toleration for dedicated=true:NoExecute can land on node-01, but the scheduler is not required to place them there. Add topologySpreadConstraints to force even distribution across all 3 nodes - the tolerated pods spread across all of them, including the dedicated one.

Steps:

• Remove the dedicated=true:NoExecute taint from node-02
• Create a Deployment named tolerated, image nginx:alpine, 3 replicas, label app: tolerated
• Add a toleration for dedicated=true:NoExecute
• Add topologySpreadConstraints with maxSkew: 1, topologyKey: kubernetes.io/hostname, whenUnsatisfiable: DoNotSchedule, labelSelector.matchLabels: app: tolerated

Task 3 - Pods with Affinity and Neutral Pods May Land on the Same Node

Remove the taint and rely on node affinity alone. affinity-pod targets node-01 via requiredDuringSchedulingIgnoredDuringExecution and lands there. Then neutral-01 - 3 replicas with topology spread and no affinity - also spreads across all 3 nodes including node-01. Affinity pulls the right pod in but does not prevent others from landing there.

Steps:

• Remove the dedicated=true:NoExecute taint from node-01
• Create a pod named affinity-pod, image nginx:alpine, with requiredDuringSchedulingIgnoredDuringExecution node affinity matching nodes labeled node-type=dedicated
• Create a Deployment named neutral-01, image nginx:alpine, 3 replicas, label app: neutral-01, with topologySpreadConstraints (maxSkew: 1, topologyKey: kubernetes.io/hostname, whenUnsatisfiable: DoNotSchedule, labelSelector.matchLabels: app: neutral-01), no affinity, no toleration

Task 4 - Neutral Pods Cannot Reach the Tainted Node

Re-apply the taint to node-01. affinity-pod is a standalone pod with no controller - NoExecute evicts and deletes it. One neutral-01 pod also goes Pending: node-01 is still counted as a topology domain with 0 pods, so placing a 3rd replica on either remaining node would create a 2-0 difference and violate maxSkew: 1. With 3 replicas and only 2 available nodes, neutral-02 pods spread across cplane-01 and node-02 only.

Steps:

• Re-apply the dedicated=true:NoExecute taint to node-01
• Create a Deployment named neutral-02, image nginx:alpine, 3 replicas, label app: neutral-02, with topologySpreadConstraints (maxSkew: 1, topologyKey: kubernetes.io/hostname, whenUnsatisfiable: DoNotSchedule, labelSelector.matchLabels: app: neutral-02), no toleration

Task 5 - Combine All Three for Full Placement Control

With the taint on node-01, deploy dedicated-pod with all three constraints: the affinity targets node-01, the toleration grants it access past the taint, and the taint keeps everything else out. Together they effectively reserve node-01 - only pods with both the matching toleration and the affinity can land there.

Steps:

• Create a pod named dedicated-pod, image nginx:alpine, with requiredDuringSchedulingIgnoredDuringExecution affinity to node-type=dedicated and a toleration for dedicated=true:NoExecute

This challenge is part of the Kubernetes Pod Scheduling skill path.