a6-plugin-ai-proxy

Configure APISIX routes to proxy LLM requests across multiple providers.

1|2|Updated Mar 6, 2026
One-click install
npx skills add https://github.com/api7/a6 --skill a6-plugin-ai-proxy-api7
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: a6-plugin-ai-proxy
Source: https://github.com/api7/a6/tree/main/skills/a6-plugin-ai-proxy
Command: npx skills add https://github.com/api7/a6 --skill a6-plugin-ai-proxy-api7

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

The ai-proxy plugin turns APISIX into an AI gateway, proxying requests to multiple LLM providers while handling authentication, endpoint routing, and streaming to deliver a unified gateway experience.

Core Features & Use Cases

  • Proxy requests to OpenAI, Azure OpenAI, Anthropic, Gemini, Vertex AI, and other providers through a single route.
  • Centralize API keys and improve observability with token and latency metrics.
  • Enable streaming, load-balancing, and failover across providers using ai-proxy-multi.

Quick Start

Create a route using the ai-proxy plugin and send a chat-completions request to test routing.

Frequently Asked Questions about a6-plugin-ai-proxy

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I configure APISIX as an AI gateway for proxying LLM requests?▼

Configure APISIX as an AI gateway by creating routes with the ai-proxy plugin, setting provider credentials, and proxying requests to multiple LLM providers through a single endpoint.

Can I route LLM traffic to multiple providers like OpenAI and Anthropic through APISIX?▼

Yes, APISIX can proxy requests to multiple LLM providers including OpenAI, Azure OpenAI, Anthropic, Gemini, and Vertex AI through a unified route while centralizing API keys.

Does the APISIX AI gateway support streaming responses for LLM providers?▼

Yes, the APISIX AI gateway supports enabling streaming for LLM requests, allowing you to handle real-time streaming responses from providers alongside standard proxy routing.

How do I set up cross-provider failover and load balancing for LLM requests?▼

Enable cross-provider routing, load-balancing, and failover across LLM providers using the ai-proxy-multi plugin to ensure high availability for proxy requests.

Can I collect observability metrics like token usage and latency for LLM requests?▼

Yes, the APISIX AI gateway improves observability by collecting token usage and latency metrics for proxied LLM requests, alongside optional logging and timeout configurations.