skills.rest
Aidas-dev
Aidas-devCOMMUNITY
Explore all skills by this author →
GitHub

kserve-operator

Create and manage KServe CRDs for model serving on Kubernetes clusters.

4|2|Updated Jun 1, 2026
Search Tags:#mlops#llm#kubernetes#vllm#model serving#kserve#inferenceservice
One-click install
npx skills add https://github.com/Aidas-dev/k8s-agent-skills --skill kserve-operator-aidas-dev
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: kserve-operator
Source: https://github.com/Aidas-dev/k8s-agent-skills/tree/main/skills/kserve-operator
Command: npx skills add https://github.com/Aidas-dev/k8s-agent-skills --skill kserve-operator-aidas-dev

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Deploying machine learning models on Kubernetes requires correctly configuring KServe CRDs like InferenceService, ServingRuntime, and InferenceGraph, which involves complex fields for storage credentials, GPU resources, canary rollouts, and autoscaling that are easy to misconfigure. ## Core Features & Use Cases - InferenceService Configuration: Generate complete InferenceService manifests with built-in predictors (SKLearn, TensorFlow, PyTorch, Triton, HuggingFace), canary traffic splitting, transformers, and explainers. - LLM and Multi-Model Serving: Configure LLMInferenceService with disaggregated prefill/decode for vLLM, ServingRuntime templates for ModelMesh, and InferenceGraph routers for ensemble, switch, and splitter topologies. - Storage and Scaling Setup: Set up S3/GCS/Azure/HuggingFace storage credentials via secrets or IRSA, Modelcar OCI mode, LocalModelCache, and Knative autoscaling parameters. - Use Case: A platform engineer needs to deploy a Llama model on GPU nodes with vLLM and tensor parallelism; this Skill produces the correct LLMInferenceService YAML with GPU limits, storage URI, and autoscaler settings. ## Quick Start Ask the agent to create a KServe InferenceService manifest that serves a scikit-learn model stored in S3 with a 10 percent canary rollout to a v2 model.

Frequently Asked Questions about kserve-operator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a KServe InferenceService for model serving?▼

Define an InferenceService with apiVersion serving.kserve.io/v1beta1 and specify a predictor such as sklearn, pytorch, or triton with a storageUri pointing to your model in S3, GCS, or a PVC. Add minReplicas, maxReplicas, and resource limits to control scaling and GPU allocation.

What is the difference between InferenceService and LLMInferenceService in KServe?▼

LLMInferenceService is a separate v1alpha1 CRD designed for large language models, supporting vLLM configuration and disaggregated prefill/decode pod topologies. InferenceService is the general v1beta1 CRD for standard predictors like SKLearn, TensorFlow, and Triton.

How do I configure S3 storage credentials for KServe?▼

Create a Secret with annotations like serving.kserve.io/s3-endpoint and serving.kserve.io/s3-region containing AWS credentials, then attach it to a ServiceAccount referenced by predictor.serviceAccountName. On AWS, you can alternatively use IRSA by annotating the ServiceAccount with an IAM role ARN.

Does KServe support canary deployments for models?▼

Yes, InferenceService supports canary rollouts via the canaryTrafficPercent field (an integer from 0 to 100) and a canary predictor block pointing to the new model version. Traffic is split between the stable and canary predictors automatically.

Why does my KServe InferenceService fail with model not found?▼

This usually means the predictor lacks a service account with valid storage credentials, so the storage initializer cannot download the model to the /models mount point. Verify the secret annotations match your storage backend and that the storageUri path is correct.

When should I use ModelMesh versus standard KServe deployment mode?▼

ModelMesh mode, enabled via the serving.kserve.io/deploymentMode annotation, uses ServingRuntime and TrainedModel resources to serve many models per pod for high-density multi-tenancy. Standard mode creates one InferenceService per model, which suits dedicated or GPU-heavy workloads.

Related Skills

Explore More Software Engineering Skills→
BagelHoleBagelHole
46

model-serving-kubernetes

Deploy and manage ML models on Kubernetes with KServe and Triton.

Community
Advanced
manastalukdarmanastalukdar
1

canary

Poll logs, error rates, and endpoints to verify post-deployment health.

Community
Advanced
villadalminevilladalmine
1

gateway

Route cluster HTTP and HTTPS traffic through a shared Cilium Gateway.

Community
Advanced
aemcoderaemcoder
2

migrate-block

Migrate a visual component into an AEM EDS block with authored HTML, scoped CSS, and preview verification.

Official
Advanced