LLM Integration Patterns

Use LiteLLM's completion or acompletion functions with a fallbacks parameter listing alternative models like GPT after Claude. The LiteLLM Router class also supports named model groups, retry counts, and timeout configuration for self-hosted deployments.

Use the Vercel AI SDK's streamText function in a route handler and return result.toUIMessageStreamResponse(). On the client, the useChat hook from @ai-sdk/react consumes the stream and updates messages in real time.

LiteLLM suits self-hosted VM backends needing full routing control, 100+ provider support, and no vendor lock-in. AI Gateway fits Vercel-hosted frontends wanting automatic failover and unified billing without managing API keys.

Define tools with the tool function using Zod schemas for inputSchema and outputSchema, plus an execute function. Pass them to streamText or an Agent with a stopWhen condition like stepCountIs to limit tool call iterations.

Use Upstash Ratelimit with a sliding window limiter backed by Redis, keyed by user ID. Check the limit before processing each request and return HTTP 429 when the daily quota is exceeded.

Use DurableAgent from Workflow DevKit when agent executions must survive crashes and redeployments, such as long-running research tasks. It provides observable steps, automatic retry on failure, and pause/resume support.