databricks-mlflow-evaluation

What problem does it solve?

This Skill streamlines the evaluation of Generative AI agents and LLM applications, enabling rigorous quality assessment, debugging, and performance optimization.

Core Features & Use Cases

• Automated Evaluation: Run mlflow.genai.evaluate() with built-in or custom scorers.
• Trace Analysis: Debug agent behavior using detailed trace data.
• Prompt Optimization: Automatically improve prompts using GEPA with aligned judges.
• Production Monitoring: Continuously score live traffic with registered scorers.
• Use Case: Evaluate a RAG agent's groundedness and relevance, then use the aligned judge and GEPA to optimize its prompt for better accuracy and reduced hallucinations.

Quick Start

Use the databricks-mlflow-evaluation skill to evaluate my agent using the safety and correctness scorers.