Challenge, Medium,  on  Kubernetes

Kubernetes Workloads: DaemonSets

A DaemonSet runs exactly one Pod on every node that matches its configuration. There is no replica count to set. When a node joins the cluster, the controller places a Pod on it. DaemonSets are used for node-level agents: log collectors, monitoring exporters, and network plugins.

Unlike Deployments, a DaemonSet has no ReplicaSets. Rollout history is stored in ControllerRevision objects instead - one per unique pod template. The controller also injects default tolerations into every Pod, covering not-ready:NoExecute, unreachable:NoExecute, disk-pressure:NoSchedule, memory-pressure:NoSchedule, pid-pressure:NoSchedule, and unschedulable:NoSchedule, so DaemonSet Pods survive node condition events and are unaffected by cordoning.


Task 1 - Create a DaemonSet

The DaemonSet controller places one Pod on each node that matches the pod template's scheduling rules. .status.desiredNumberScheduled reports the target node count and .status.numberReady reports how many of those Pods are currently ready.

Steps:

  • Create a DaemonSet named node-agent with selector app: node-agent, one container named agent using image nginx

Run kubectl get ds node-agent - DESIRED and READY both show 3. Run kubectl get pods -l app=node-agent -o wide to confirm each Pod landed on a different node. Pod names follow the pattern node-agent-<suffix>. Run kubectl describe pod <pod-name> and look at the Tolerations section - the six default tolerations appear even though the manifest declared none.


Task 2 - Roll an Image Update

A DaemonSet's default update strategy is RollingUpdate. When the pod template changes, the controller replaces Pods one node at a time by default, controlled by .spec.updateStrategy.rollingUpdate.maxUnavailable.

Steps:

  • Update the node-agent image to nginx:alpine

Run kubectl get ds node-agent - DESIRED, CURRENT, READY, and UP-TO-DATE all show 3. Run kubectl get pods -l app=node-agent -o yaml | grep "image:" to confirm every Pod is running nginx:alpine. The UP-TO-DATE column reaching 3 marks the rollout complete.

Each pod template change creates a ControllerRevision object that stores the spec at that point. Run kubectl get controllerrevision -l app=node-agent - two revisions now exist, one for the initial nginx template and one for nginx:alpine. Run kubectl get controllerrevision <name> -o yaml to inspect the .data field, which holds the patch applied at that revision.


Task 3 - Roll Back the Update

kubectl rollout undo finds the target ControllerRevision and restores the pod template from it. The rolling update strategy then replaces the Pods.

Steps:

  • Roll back node-agent to the previous revision

Run kubectl rollout history ds/node-agent - revision 1 is gone and revision 3 now points to the nginx template. The controller reused the existing ControllerRevision object and renumbered it from 1 to 3 rather than creating a new one. Run kubectl get controllerrevision -l app=node-agent - two objects remain, numbered 2 and 3.

To target a specific revision rather than the previous one, pass --to-revision=<n> - for example kubectl rollout undo ds/node-agent --to-revision=2. The revision number maps directly to a ControllerRevision object.


Task 4 - Limit Coverage with nodeSelector

A nodeSelector in the pod template restricts which nodes the DaemonSet targets. Nodes whose labels do not match every entry in .spec.template.spec.nodeSelector are excluded from .status.desiredNumberScheduled, regardless of taints or tolerations.

nodeAffinity provides the same targeting capability but with operators (In, NotIn, Exists, DoesNotExist) and supports both required (requiredDuringSchedulingIgnoredDuringExecution) and preferred (preferredDuringSchedulingIgnoredDuringExecution) scheduling rules.

Steps:

  • Add the label monitoring=enabled to node-01
  • Edit the node-agent DaemonSet and add .spec.template.spec.nodeSelector: {monitoring: "enabled"}

Run kubectl get ds node-agent - DESIRED shows 1. Run kubectl get pods -l app=node-agent -o wide to confirm the single Pod landed on node-01. cplane-01 and node-02 are excluded because their labels do not match the selector - not by any taint or toleration.