KServe v0.19 Install Guide: Kubernetes ML Serving Done Right

Most tutorials install KServe wrong. They walk you through Knative mode because that’s what the quickstart script does – and if you’re planning to serve an LLM, that’s the setup you probably don’t want. This guide leads with Standard Kubernetes mode. Knative gets covered too, but only after you understand what you’re trading away.

KServe (v0.19.0, released June 12, 2026 – verify on the releases page before running any of these commands) is a CNCF incubating project for serving both predictive and generative ML models on Kubernetes. The two deployment modes – Standard and Knative – look almost identical from the outside. The differences only surface when something goes wrong, or when you’re six nodes deep into a GPU cluster wondering why your model pod won’t cold-start.

Standard vs. Knative: pick this before you install anything

Standard mode is for LLMs. Knative is for burst-y predictive workloads. That’s the short version – per the KServe admin overview, Standard handles GPU-accelerated models, long-running requests, and streaming responses. Knative handles bursty traffic patterns where pods need to scale to zero between requests.

Scale-to-zero is gone in Standard mode. The docs are direct about it: HTTP requests can’t trigger a cold start in Standard (as of v0.19). KEDA can autoscale on custom metrics, but the floor is one pod, not zero. For a 70B parameter model burning $8/hr on a GPU node, that matters. Decide now, not after install.

Think of it like choosing between a sports car and a city bus. The bus (Knative) scales up when passengers arrive and parks when empty – efficient for unpredictable crowds. The sports car (Standard) stays running and ready, burns fuel at idle, but handles the long haul without making passengers wait for it to warm up.

System requirements

GPU nodes aren’t listed here because the requirement depends entirely on your models. Predictive-only workloads (sklearn, ONNX, XGBoost) run fine on CPU. Anything LLM-shaped needs GPU – plan the node pool before you get to the InferenceService YAML.

Install KServe v0.19.0 in Standard mode

Three paths exist: raw YAML, quick-install script, Helm. Helm survives upgrades. YAML shows you what’s actually deployed. Pick one.

Step 1 – Install cert-manager

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/latest/download/cert-manager.yaml
kubectl wait --for=condition=Available deployment --all -n cert-manager --timeout=300s

Step 2 – Install Gateway API CRDs

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml

Gateway API is a separate install – it’s not shipped with Kubernetes itself. After this, pick a network controller. Envoy Gateway and Istio are the common choices (check the KServe networking docs for the current compatibility matrix, as supported controllers change between releases).

Step 3 – Install KServe

--server-side is not optional. Without it, kubectl stores the entire CRD in an annotation and blows past etcd’s 262144-byte limit. Per the KServe deployment docs: the InferenceService CRD is large enough that client-side apply will fail every time.

# Install KServe CRDs and controller
kubectl apply --server-side -f https://github.com/kserve/kserve/releases/download/v0.19.0/kserve.yaml
kubectl apply --server-side -f https://github.com/kserve/kserve/releases/download/v0.19.0/kserve-cluster-resources.yaml

# Switch default deployment mode to Standard
kubectl patch configmap/inferenceservice-config -n kserve --type=strategic 
 -p '{"data": {"deploy": "{"defaultDeploymentMode": "Standard"}"}}'

# Enable Gateway API
kubectl patch configmap/inferenceservice-config -n kserve --type=strategic 
 -p '{"data": {"ingress": "{"enableGatewayApi": true, "kserveIngressGateway": "kserve/kserve-ingress-gateway"}"}}'

Helm alternative – note the version in the facts array references v0.18.0; verify the correct chart version tag for v0.19.0 on the releases page before running:

helm install kserve oci://ghcr.io/kserve/charts/kserve-resources --version v0.19.0 
 --set kserve.controller.deploymentMode=Standard 
 --set kserve.controller.gateway.ingressGateway.className=istio

Serving generative AI? Install the LLMInferenceService controller too – it handles KV-cache management, multi-node serving, and AI gateway integration. One limitation as of v0.19: LocalModel (the model cache component) only supports InferenceService workloads. LLMInferenceService caching support is planned but not yet shipped.

Verification

kubectl get pods -n kserve
# Expected:
# kserve-controller-manager-xxxxxxxx-xxxxx 2/2 Running 0 38s

Skip the sklearn iris demo. Deploy something closer to your actual workload:

cat <<EOF | kubectl apply -f -
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
 name: qwen-tiny
spec:
 predictor:
 model:
 modelFormat:
 name: huggingface
 storageUri: "hf://Qwen/Qwen2.5-0.5B-Instruct"
 resources:
 requests:
 cpu: "1"
 memory: 4Gi
EOF

curl -v -H "Host: qwen-tiny.default.example.com" 
 http://localhost:8080/openai/v1/chat/completions 
 -d '{"model":"qwen-tiny","messages":[{"role":"user","content":"hi"}]}'

The response comes back in OpenAI-compatible format – same protocol, same client code. That’s been the direction since the vLLM integration landed (v0.15 added Qwen3 and Llama4 support alongside vLLM 0.8.5, per the KServe blog).

Three errors. All real.

• failed calling webhook "webhook.cert-manager.io": connect: connection refused
  Not a config bug – a timing one. The cert-manager webhook performs leader election at startup and takes several seconds before it’s reachable. Your KServe apply ran before that window closed. Fix: the kubectl wait in Step 1 exists precisely for this. If you skipped it, run it now and retry. The cert-manager troubleshooting docs describe this exact sequence.
• failed calling webhook "isvc.serving.kserve.io"
  KServe’s own webhook isn’t ready or its certs haven’t propagated. Check kubectl get pods -n kserve, verify cert-manager generated valid certificates, and read the controller pod logs – that sequence is from the official KServe developer guide.
• metadata.annotations: Too long: must have at most 262144 bytes
  Forgot --server-side. Re-run the apply with it. Client-side apply stores the full CRD in an annotation; the InferenceService CRD is too large for that path.

Upgrading from v0.17 or earlier

v0.17 Helm users: read the release notes first. That release introduced a major Helm chart restructuring as a breaking change – chart names moved. A straight helm upgrade against your old release name will either fail or silently reset config. Check the v0.17.0 release notes for the migration steps before touching anything.

Uninstall order matters too:

# Delete InferenceServices first - controller needs to clean up owned resources
kubectl delete inferenceservices --all -A

kubectl delete -f https://github.com/kserve/kserve/releases/download/v0.19.0/kserve-cluster-resources.yaml
kubectl delete -f https://github.com/kserve/kserve/releases/download/v0.19.0/kserve.yaml

kubectl delete namespace kserve

Delete CRDs before InferenceServices and you’ll have orphaned resources with finalizers blocking namespace deletion. Kubernetes gotcha, but easy to hit here specifically.

FAQ

Do I need Istio?

No. Gateway API is the preferred abstraction now, and multiple controllers work with it. Istio is common but not mandatory.

Can I test KServe locally before hitting a real cluster?

Yes – kind or minikube with the quickstart script works. Allocate at least 4 CPU and 8 GiB to the local cluster (practical minimum, not an official spec). One thing to know about the Qwen 0.5B model specifically: it’ll technically run on CPU, but generation is slow enough that you’ll think the pod is hung. Use a scikit-learn or ONNX predictor for local smoke tests and save the LLM examples for a cluster with GPU access.

What’s the actual difference between InferenceService and LLMInferenceService?

InferenceService is the general CRD – any framework, any model type. LLMInferenceService is purpose-built for generative AI: KV-cache management, multi-node serving, AI gateway integration. A common misconception is that you need LLMInferenceService to serve any language model – you don’t. A smaller model like Qwen2.5-0.5B deploys fine through the standard CRD. LLMInferenceService becomes relevant when you’re coordinating multi-node GPU serving for something like Llama-4, or when you need the cache management features that don’t exist in the base controller.

Next step: spin up a kind cluster, run the three install commands, deploy the Qwen tiny model. Pod reaches Ready and curl returns tokens? Your install is production-shaped. Everything past this is scaling and networking config.