Challenge Easy

Safe Node Drain With a Disruption Budget

Omkar Shelke
by  Omkar Shelke · on
Kubernetes
node-01 is due for maintenance, and it hosts two of the three checkout-api replicas. Add a PodDisruptionBudget so the drain can never take checkout below two available replicas, then drain the node.
#cka#pdb#poddisruptionbudget#drain#maintenance

Scenario

checkout-api in the shop namespace takes every payment on the storefront. It runs three replicas, and the team's rule is simple: checkout never runs on fewer than two.

node-01 is scheduled for a kernel patch tonight. It currently hosts two of the three replicas:

kubectl get pods -n shop -o wide

Today nothing limits how many checkout-api replicas a drain may evict at once. If node-01 were drained as things stand, two of the three replicas would go down together, and checkout would run on one replica until their replacements became Ready.

Task

  1. Create a PodDisruptionBudget named checkout-api-pdb in the shop namespace that keeps at least 2 of the checkout-api replicas available.
  2. Drain node-01 for maintenance once checkout-api-pdb is in place.
  3. Keep node-01 cordoned after the drain, because the maintenance window is still open. The test passes once checkout-api has 3 available replicas again on the other nodes.
Important

Do not edit, scale, or delete the checkout-api Deployment or its Pods by hand. The drain has to do the work, and the budget has to control its pace.

Hint 1 | The shape of a PodDisruptionBudget

A budget selects Pods by label and states how many must stay up, or how many may be down, during a voluntary disruption.

kubectl get pods -n shop --show-labels
kubectl explain poddisruptionbudget.spec
kubectl create poddisruptionbudget --help

Once it exists, check that it counts the Pods you expect:

kubectl get pdb -n shop
kubectl describe pdb <name> -n shop
Hint 2 | Draining a node

A drain cordons the node first, then evicts its Pods through the Eviction API, which is the API a budget guards. Some system Pods on the node can stop it before any eviction starts. The error names each one and the flag it needs:

kubectl drain --help
kubectl drain <node> <flags>

A drain that stops with an error has still cordoned the node.

Hint 3 | Reading what the drain prints

A refused eviction is not a failure. The drain retries every few seconds until the budget allows the next one. Watch the budget from a second terminal while the drain runs:

kubectl get pdb -n shop -w
kubectl get pods -n shop -o wide -w

⚒ Test Cases

How to Author Challenges on iximiuz Labs

Instead of providing a subpar online editing experience, iximiuz Labs offers a helper CLI tool called labctl, allowing you to use your favorite text editor (or a full-featured IDE) to write content from the comfort of your local machine.

Install labctl CLI

curl -sf https://labs.iximiuz.com/cli/install.sh | sh

This will download and install the latest version of the labctl CLI. You only need to do this once per workstation.

Authorize labctl

labctl auth login

This will open a browser window asking you to authorize labctl to access your account. You need to do it after a fresh install of labctl and repeat it whenever the auth session expires.

Pull challenge content

labctl content pull challenge safe-node-drain-with-a-disruption-budget-8fc8f25e

This will create a local copy of the challenge content in a directory named safe-node-drain-with-a-disruption-budget-8fc8f25e. You only need to do this once per challenge.

Stream your changes

labctl content push -fw challenge safe-node-drain-with-a-disruption-budget-8fc8f25e

Run this command in a separate terminal to continuously upload your changes to the server while editing the challenge in your favorite text editor or IDE.

You can also use labctl to create, list, and delete your content. Learn more about the available commands: labctl content --help

Start challenge