What problem does it solve? Debugging and monitoring LLM applications in production is difficult without visibility into traces, evaluation results, and system behavior. This Skill provides a self-hosted observability platform for tracing LLM calls, running systematic evaluations, and monitoring production AI systems without vendor lock-in. ## Core Features & Use Cases - OpenTelemetry Tracing: Instrument OpenAI, LangChain, LlamaIndex, and Anthropic SDKs to capture detailed traces and spans of every LLM call. - LLM Evaluation Framework: Run built-in evaluators (hallucination, relevance, toxicity) or custom LLM-as-judge evaluators on datasets and production spans. - Datasets & Experiments: Create versioned test datasets and run experiments to compare prompts, models, and configurations with aggregate metrics. - Use Case: A team deploying a RAG chatbot instruments their LangChain pipeline with Phoenix, runs hallucination evaluations on production spans nightly, and compares prompt variants via experiments before each release. ## Quick Start Install arize-phoenix, launch the Phoenix server, and instrument my OpenAI client so all LLM calls are traced to the local Phoenix UI.
