What problem does it solve? Debugging and evaluating LLM applications is difficult without visibility into prompts, chains, and agent executions. This Skill provides tracing, dataset-based evaluation, and production monitoring workflows for LLM systems using LangSmith. ## Core Features & Use Cases - Automatic Tracing: Capture inputs, outputs, latency, and token usage for LLM calls using the @traceable decorator or the OpenAI wrapper. - Dataset Evaluation: Build test datasets from production traces and run custom or built-in evaluators to measure output quality. - Production Monitoring: Track errors, costs, feedback scores, and latency trends across projects, with sampling and batching controls. - Use Case: A team shipping a RAG chatbot traces every retrieval and generation step, builds a regression dataset from flagged production runs, and runs automated evaluations in CI to catch accuracy drops before deployment. ## Quick Start Set the LANGSMITH_API_KEY and LANGSMITH_TRACING environment variables, then ask the assistant to add @traceable decorators to your LLM functions so their calls appear in the LangSmith dashboard.
