skills.rest
merceralex397-collab
merceralex397-collabCOMMUNITY
Explore all skills by this author →
GitHub

airunway-aks-setup

Guides end-to-end AI Runway installation and first model deployment on AKS clusters.

Updated Jul 20, 2026
Search Tags:#kubernetes#gpu#model deployment#aks#llm inference#ai runway#kaito
One-click install
npx skills add https://github.com/merceralex397-collab/alex-stack --skill airunway-aks-setup-merceralex397-collab
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: airunway-aks-setup
Source: https://github.com/merceralex397-collab/alex-stack/tree/main/dev/azure-profile/azure-skills-main/azure-skills-main/.github/plugins/azure-skills/skills/airunway-aks-setup
Command: npx skills add https://github.com/merceralex397-collab/alex-stack --skill airunway-aks-setup-merceralex397-collab

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Setting up AI model serving on a bare AKS cluster involves many error-prone steps: verifying cluster state, installing the AI Runway controller and CRDs, assessing GPU compatibility, choosing an inference provider, and deploying a first model. This Skill walks users through the entire onboarding sequence with verification and error handling at each stage. ## Core Features & Use Cases - Six-Step Guided Workflow: Cluster verification, controller installation, GPU assessment, provider setup, first deployment, and smoke test, with support for resuming via skip-to-step. - GPU Compatibility Intelligence: Matches detected GPUs (T4, V100, A100, H100, etc.) against known profiles, flagging dtype and attention-backend constraints before deployment. - Provider Recommendation: Recommends and installs KAITO, Dynamo, or KubeRay based on hardware and workload, with model sizing guidance for VRAM capacity. - Use Case: A platform engineer with a fresh AKS cluster and A100 nodes uses this Skill to install the AI Runway controller, register KAITO, and deploy Llama-3.1-8B-Instruct with a HuggingFace token secret, then smoke-tests the endpoint. ## Quick Start Ask the assistant to set up AI Runway on your existing AKS cluster and deploy your first model, optionally specifying a step number to resume from.

Frequently Asked Questions about airunway-aks-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up AI Runway on an AKS cluster?▼

Follow the six-step workflow: verify cluster connection and GPUs, install the controller with make controller-install and controller-deploy, assess GPU compatibility, install an inference provider, apply a ModelDeployment CR, and smoke test the endpoint. You can resume from any step with skip-to-step N.

Which inference provider should I use for model serving on AKS?▼

KAITO is the default recommendation for most users, covering CPU-only inference via llama.cpp and GPU inference via vLLM. Choose Dynamo for high-throughput serving with separate prefill and decode phases, or KubeRay if your team already uses Ray.

Does AI Runway work on AKS clusters without GPUs?▼

Yes, CPU-only clusters are supported through KAITO with llama.cpp. The recommended starter model is google/gemma-3-1b-it-qat-q8_0-gguf, a GGUF Q8 quantized model that runs on CPU.

Why do I get bfloat16 errors when running inference on T4 or V100 GPUs?▼

T4 and V100 GPUs do not support bfloat16, causing errors or silent dtype casting at inference time. Add --dtype float16 to the serving arguments, and on V100 use the xformers attention backend since flash attention support is limited.

How do I deploy gated HuggingFace models like Llama on AKS?▼

Create a Kubernetes secret named hf-token containing your HuggingFace access token, then reference it in the ModelDeployment CR via huggingFaceTokenSecretRef. Non-gated alternatives like Phi-3 and Gemma models require no token.

Why is my ModelDeployment stuck in Pending status?▼

Pending status usually indicates GPU scheduling failure or a provider that is not ready. Run kubectl describe modeldeployment to inspect events, and verify the provider shows status.ready true. Large models like 70B may also take 20-40 minutes downloading weights.

Related Skills

Explore More Software Engineering Skills→
theslashdojotheslashdojo

aks

Create and manage AKS clusters, sync kubeconfig credentials, and add node pools via Azure CLI.

Official
Intermediate
JosephWoodallJosephWoodall
1

modal

Deploy Python AI workloads on Modal's serverless GPU infrastructure.

Community
Advanced
googlegoogle
19.1k

gke-workload-troubleshooting

Diagnoses GKE workload failures via pod status, events, and logs, then proposes GitOps manifest fixes.

Official
Advanced
rrbandarrbanda

airunway-aks-setup

Automate AI Runway deployment on AKS from bare cluster to running model.

Community
Advanced