Evals

What problem does it solve?

This Skill provides a robust framework for evaluating AI agents, ensuring their performance meets predefined quality standards and identifying regressions before they impact users.

Core Features & Use Cases

• Objective Evaluation: Utilizes code-based, model-based, and human graders for comprehensive assessment.
• Workflow Testing: Evaluates entire agent interactions, not just single outputs.
• Use Case: Automatically test if a new version of your customer service agent correctly handles common user queries, provides accurate information, and maintains a helpful tone, flagging any performance dips.

Quick Start

Run the evals skill to evaluate the current agent's performance on the core behaviors suite.