eval-harness

You can run regression and capability evaluations for Claude Code sessions by defining eval files in .claude/evals and executing /eval check to validate behavior and generate standardized reports.

Eval-driven development for Claude Code involves defining evaluations before coding, executing capability and regression checks, and monitoring pass@k metrics to ensure feature validity and prevent behavioral regressions.

To generate standardized evaluation reports for Claude Code features, implement your eval tests in a markdown file under .claude/evals, then execute the /eval report command to produce a summary of pass@k metrics.

Yes, the eval framework supports pass@k and pass^k scoring metrics, alongside code-based, model-based, and human grader types to provide robust assessment for Claude Code capability and regression evaluations.

Using markdown files under .claude/evals for regression evals requires predefined eval definitions and tests before execution, meaning evaluations must be codified manually rather than generated dynamically during the Claude Code session.