Challenge, Medium,  on  Kubernetes

Validate CSI Storage Performance Using FIO and a Kubernetes Job

Omkar Shelke
by  Omkar Shelke

Scenario

You are working on a Kubernetes cluster where storage performance needs to be validated using a CSI-backed StorageClass. The cluster has the local-path StorageClass available, backed by the local-path-provisioner CSI driver.

Before deploying stateful workloads, the team needs to benchmark the underlying storage performance — specifically random write I/O — using FIO (Flexible I/O Tester), a standard Linux storage benchmarking tool.

Task

Create a PersistentVolumeClaim named app-data-pvc in the storage-lab namespace using the StorageClass local-path with a requested size of 1Gi.

Then create a Job named storage-check in the storage-lab namespace using the image lfedge/eden-fio-tests:1b1b353 that:

  • Mounts the PVC at /data using a volume named data-vol
  • Is scheduled on node cplane-01 using nodeSelector with key kubernetes.io/hostname
  • The Job runs only once.
  • The Job does not restart after completion.
  • Runs the following FIO benchmark command:
fio --name=randwrite --filename=/data/testfile --size=1G --bs=4k --rw=randwrite --runtime=25 --time_based --direct=1 --output=/data/fio-result.txt && cat /data/fio-result.txt

After creating the resources, the FIO benchmark runs for 25 seconds — the Job will take approximately 1 minute to fully complete.

Wait until the Pod shows Completed status before checking the logs or verifying the report file.

Use kubectl get pod -n storage-lab -l job-name=storage-check -w to watch the Pod status in real time.

Hint

Create a PVC in the storage-lab namespace with storageClassName: local-path and accessModes: ReadWriteOnce. ReadWriteOnce is the only supported access mode for local-path because the volume is stored directly on the local disk of cplane-01 — only one node can mount it at a time.

Set completions: 1 and restartPolicy: Never to ensure the Job runs once and does not retry. Use nodeSelector with key kubernetes.io/hostname to pin the Pod to cplane-01. Name the volume data-vol and mount it at /data. Use --output=/data/fio-result.txt && cat /data/fio-result.txt so the report is both saved on the PVC and visible in kubectl logs.

See the official docs:

Test Cases

How to Author Challenges on iximiuz Labs

Instead of providing a subpar online editing experience, iximiuz Labs offers a helper CLI tool called labctl, allowing you to use your favorite text editor (or a full-featured IDE) to write content from the comfort of your local machine.

Install labctl CLI

curl -sf https://labs.iximiuz.com/cli/install.sh | sh

This will download and install the latest version of the labctl CLI. You only need to do this once per workstation.

Authorize labctl

labctl auth login

This will open a browser window asking you to authorize labctl to access your account. You need to do it after a fresh install of labctl and repeat it whenever the auth session expires.

Pull challenge content

labctl content pull challenge validate-csi-storage-performance-using-fio-and-k8s-job-752fe399

This will create a local copy of the challenge content in a directory named validate-csi-storage-performance-using-fio-and-k8s-job-752fe399. You only need to do this once per challenge.

Stream your changes

labctl content push -fw challenge validate-csi-storage-performance-using-fio-and-k8s-job-752fe399

Run this command in a separate terminal to continuously upload your changes to the server while editing the challenge in your favorite text editor or IDE.

You can also use labctl to create, list, and delete your content. Learn more about the available commands: labctl content --help

Start challenge