What problem does it solve? AI-assisted coding sessions often lack objective success criteria, making it hard to know whether a change actually works or silently broke existing behavior. This Skill provides a formal evaluation framework that treats evals as unit tests for AI development, so every task has measurable pass/fail outcomes. ## Core Features & Use Cases - Capability and Regression Evals: Define what Claude should newly accomplish and verify existing functionality still passes after changes. - Multiple Grader Types: Use deterministic code-based graders (grep, npm test, build checks), model-based graders for open-ended outputs, or flag items for human review. - Reliability Metrics: Track pass@k and pass^k metrics to measure agent reliability across attempts and model versions. - Use Case: Before adding authentication to an app, define evals for registration, login, and session handling, implement the feature, then run the evals and generate a report showing 5/5 capability evals passed with pass@3 at 100%. ## Quick Start Ask the agent to define an eval-driven development checklist with pass/fail criteria for the feature you are about to implement.
