Course
Generative AI in Kubernetes
A hands-on workshop covering the state-of-the-art for running generative AI workloads on Kubernetes. Deploy LLMs with Ollama, use KServe for model serving, explore GPU scheduling with fake-gpu-operator and DRA, configure Gateway API Inference Extension, and learn about KAITO and llm-d. All exercises run on CPU — no GPU required.
This course is currently broken
It no longer works as intended - most likely, a tool, an image, or an API it relies on has changed since it was written.
About This Course
Workshop Overview
Duration: 90 minutes Level: Intermediate (comfortable with kubectl, YAML, Helm) Event: Devoxx Greece 2026
What You'll Build
Starting from a bare Kubernetes cluster, you'll build a complete AI inference platform:
| Lab | Topic |
|---|---|
| Lab 1 | Cluster Setup + fake-gpu-operator |
| Lab 2 | Deploy an LLM (Ollama + TinyLlama) |
| Lab 3 | KServe InferenceService |
| Lab 4 | GPU Scheduling & DRA |
| Lab 5 | Gateway API Inference Extension |
| Lab 6 | KAITO Workspace |
| Lab 7 | llm-d Disaggregated Inference |
Author
Alessandro Vozza — Cloud Native architect, Golden Kubestronaut, KubeCon speaker
What's Inside?
About the author
