ai-llm-inference
CommunityHigh-throughput, cost-aware LLM inference.
Authorvasilyu1983
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Optimizes inference with vLLM, quantization, and hardware-aware serving.
Core Features & Use Cases
- Throughput optimization: vLLM, continuous batching.
- Quantization: FP8/FP4 to cut costs with minimal accuracy loss.
- Serving: GPU-accelerated and CPU options for different scales.
Quick Start
Configure a vLLM-based serving pipeline with FP8 quantization for a 70B model.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: ai-llm-inference Download link: https://github.com/vasilyu1983/AI-Agents-public/archive/main.zip#ai-llm-inference Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.