Skills
.Work. You
Rest

routing-architecture

Community

Routing strategies for fast, healthy LLM routing.

Software Engineering#routing#LLM#latency#Rust#health-check#load-balancing#DashMap
Authormajiayu000
Version1.0.0
Installs0

System Documentation

What problem does it solve?

LiteLLM-RS routing architecture delivers a lock-free, health-aware provider selection mechanism to maximize throughput and reliability in multi-provider LLM deployments.

Core Features & Use Cases

  • Seven routing strategies (SimpleShuffle, RoundRobin, LeastBusy, LatencyBased, CostBased, UsageBased, RateLimitAware) enable flexible load distribution across 66+ providers.
  • Built-in health checks, fallback chains, and configurable policies maintain service continuity during provider failures and outages.
  • Common use cases include latency-optimized routing, cost-aware provider selection, quota management, and rate-limit aware routing under high concurrency.

Quick Start

Configure your router with a chosen strategy (e.g., latency_based), list the providers, and enable health checks to begin routing across providers.

Dependency Matrix

Required Modules

None required

Components

Standard package

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: routing-architecture
Download link: https://github.com/majiayu000/litellm-rs/archive/main.zip#routing-architecture

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.