skills.rest
wwwillott
wwwillottCOMMUNITY
Explore all skills by this author →
GitHub

serving-llms-vllm

Serve scalable LLM inference with vLLM for OpenAI-compatible APIs.

Updated Aug 23, 2026
Search Tags:#kubernetes#docker#quantization#inference#production-deployment#vllm#llm-serving
One-click install
npx skills add https://github.com/wwwillott/jobnimbus --skill serving-llms-vllm-wwwillott
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: serving-llms-vllm
Source: https://github.com/wwwillott/jobnimbus/tree/main/skills/mlops/inference/vllm
Command: npx skills add https://github.com/wwwillott/jobnimbus --skill serving-llms-vllm-wwwillott

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Provision scalable, production-grade LLM serving using vLLM to expose OpenAI-compatible APIs and efficient quantization.

Core Features & Use Cases

  • High-throughput LLM serving with vLLM for OpenAI-compatible endpoints.
  • Quantization support (AWQ, GPTQ, FP8) to fit large models on limited GPUs.
  • Production deployment patterns with Docker, Kubernetes, and robust monitoring.

Quick Start

Launch a minimal vLLM server with a small model and send a test prompt to verify end-to-end deployment.

Frequently Asked Questions about serving-llms-vllm

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy an LLM for production inference using vLLM and Kubernetes?▼

Deploy production LLM inference using vLLM by packaging the server in Docker containers and orchestrating scalable deployments on Kubernetes, exposing OpenAI-compatible API endpoints with robust monitoring.

What quantization methods does vLLM support for fitting large models on limited GPUs?▼

vLLM supports AWQ, GPTQ, and FP8 quantization methods to reduce model memory footprints. This enables fitting large language models onto limited GPUs while tuning for optimal latency and throughput.

Can I expose OpenAI-compatible API endpoints using vLLM?▼

Yes, you can expose OpenAI-compatible API endpoints using vLLM. This allows existing applications built for the OpenAI API standard to seamlessly interact with your self-hosted, production-grade LLM serving infrastructure.

How do I tune vLLM for better latency and throughput in production pipelines?▼

Tune vLLM latency and throughput in production pipelines by applying quantization techniques like AWQ, GPTQ, or FP8. Combine this with Kubernetes scaling and performance monitoring to optimize API response times.

What is the best way to start a minimal vLLM server for testing?▼

Start a minimal vLLM server for testing by launching it with a small model. You can then send a test prompt to verify that your end-to-end deployment and OpenAI-compatible API are functioning correctly.

Related Skills

Explore More Software Engineering Skills→
TehminanazTehminanaz
1

kafka-kraft-strimzi

Deploy and tune Kafka in KRaft mode with Strimzi on resource-constrained Kubernetes clusters.

Community
Advanced
qcmuuqcmuu

serving-llms-vllm

Deploy vLLM inference endpoints with quantization and tensor parallelism.

Community
Advanced
GregsGreyCodeGregsGreyCode
6

serving-llms-vllm

Deploy vLLM servers with PagedAttention, continuous batching, and OpenAI-compatible endpoints.

Community
Advanced
anilcan-karaanilcan-kara

serving-llms-vllm

Serve LLMs with an OpenAI-compatible API using vLLM.

Community
Advanced