Deploy AI Model Serving on Kubernetes with KServe and Helm
Scenario
Your team is building an AI model serving platform on Kubernetes. KServe is the chosen framework — it provides a standardised, scalable way to deploy machine learning models as inference services on Kubernetes, supporting frameworks like TensorFlow, PyTorch, Triton, and large language models.
Before any model can be deployed, the KServe infrastructure must be set up on the cluster. cert-manager is already installed and running — it is a required dependency that KServe uses to manage its webhook TLS certificates.
Task
Set up KServe v0.18.0 on this cluster using Helm so the team can start
deploying AI inference services. Install the required Helm charts from
oci://ghcr.io/kserve/charts/ in order, grouped below into three stages:
Install CRDs, Deploy Controller, and Register Runtimes.
Registers the Custom Resource Definitions KServe needs — InferenceService,
ServingRuntime, and the LLM inference service CRDs. Install both charts
below into the kserve namespace before anything else. Use --create-namespace
so Helm creates the namespace automatically.
KServe CRDs
| Field | Value |
|---|---|
| Chart | oci://ghcr.io/kserve/charts/kserve-crd |
| Release name | kserve-crd |
| Namespace | kserve |
| Version | v0.18.0 |
LLM Inference Service CRDs
| Field | Value |
|---|---|
| Chart | oci://ghcr.io/kserve/charts/kserve-llmisvc-crd |
| Release name | kserve-llmisvc-crd |
| Namespace | kserve |
| Version | v0.18.0 |
After installation verify the stack is healthy:
helm list -A
kubectl get pods -n kserve
kubectl get clusterservingruntimes
Hint 1 — Install CRDs First Using OCI Charts
KServe charts live on an OCI registry — no helm repo add needed. Here
is how an OCI-based Helm install looks in general:
helm install my-release oci://ghcr.io/some-org/charts/some-chart \
--version v1.0.0 \
--namespace my-namespace \
--create-namespace
You need to install two CRD charts before anything else. Check the task
for the correct release names, chart names, version, and namespace.
The --create-namespace flag creates the namespace automatically so you
don't need a separate kubectl create namespace step.
Documentation
Hint 2 — Deploy Both Controllers with the Right Flags
The --set flag overrides a value inside the chart's default configuration,
and --wait tells Helm to block until all Deployments are Available before
returning. For example:
helm install my-release oci://ghcr.io/some-org/charts/some-chart \
--version v1.0.0 \
--namespace my-namespace \
--set some.feature.mode=Standard \
--set some.feature.disabled=true \
--wait
Install kserve-resources first — it needs three --set flags to disable
Istio and Knative features. Once that controller is ready, install
kserve-llmisvc-resources — it needs one --set flag. Check the task
for the exact flag names and values for both charts.
Documentation
Hint 3 — Install Runtime Configs and Verify
The final chart (kserve-runtime-configs) goes into the kserve namespace.
It needs two --set flags to enable both traditional ML runtimes and LLM
runtimes — check the task for the flag names. The install pattern is the
same as before:
helm install my-release oci://ghcr.io/some-org/charts/some-chart \
--version v1.0.0 \
--namespace my-namespace \
--set some.config.enabled=true \
--set some.other.enabled=true
Then verify the full stack:
helm list -A
kubectl get pods -n _____
kubectl get _____
Documentation