Course

Generative AI in Kubernetes

Alessandro Vozza
by  Alessandro Vozza · on
Kubernetes

A hands-on workshop covering the state-of-the-art for running generative AI workloads on Kubernetes. Deploy LLMs with Ollama, use KServe for model serving, explore GPU scheduling with fake-gpu-operator and DRA, configure Gateway API Inference Extension, and learn about KAITO and llm-d. All exercises run on CPU — no GPU required.

#ai#inference#gpu#vllm#kserve
Generative AI in Kubernetes (cover image)

Reset Progress

Are you sure you want to reset your progress for this course? This action cannot be undone.

About This Course

Workshop Overview

Duration: 90 minutes Level: Intermediate (comfortable with kubectl, YAML, Helm) Event: Devoxx Greece 2026

What You'll Build

Starting from a bare Kubernetes cluster, you'll build a complete AI inference platform:

LabTopic
Lab 1Cluster Setup + fake-gpu-operator
Lab 2Deploy an LLM (Ollama + TinyLlama)
Lab 3KServe InferenceService
Lab 4GPU Scheduling & DRA
Lab 5Gateway API Inference Extension
Lab 6KAITO Workspace
Lab 7llm-d Disaggregated Inference

Author

Alessandro Vozza — Cloud Native architect, Golden Kubestronaut, KubeCon speaker

What's Inside?

About the Author

Alessandro Vozza

Alessandro Vozza

Find this author online

How to Author Courses on iximiuz Labs

Instead of providing a subpar online editing experience, iximiuz Labs offers a helper CLI tool called labctl, allowing you to use your favorite text editor (or a full-featured IDE) to write content from the comfort of your local machine.

Install labctl CLI

curl -sf https://labs.iximiuz.com/cli/install.sh | sh

This will download and install the latest version of the labctl CLI. You only need to do this once per workstation.

Authorize labctl

labctl auth login

This will open a browser window asking you to authorize labctl to access your account. You need to do it after a fresh install of labctl and repeat it whenever the auth session expires.

Pull course content

labctl content pull course genai-k8s-workshop-fb58e264

This will create a local copy of the course content in a directory named genai-k8s-workshop-fb58e264. You only need to do this once per course.

Stream your changes

labctl content push -fw course genai-k8s-workshop-fb58e264

Run this command in a separate terminal to continuously upload your changes to the server while editing the course in your favorite text editor or IDE.

You can also use labctl to create, list, and delete your content. Learn more about the available commands: labctl content --help