Define capability and regression evals with code-based, model-based, or human graders.
npx skills add https://github.com/kouiso/designdiff --skill eval-harness-kouiso
Please help me install this Agent Skill. Skill: eval-harness Source: https://github.com/kouiso/designdiff/tree/main/.claude/skills/eval-harness Command: npx skills add https://github.com/kouiso/designdiff --skill eval-harness-kouiso
This Skill provides a structured framework for evaluating AI code sessions, enabling rigorous testing and performance measurement through eval-driven development (EDD) principles.
Define a new capability evaluation for a feature named 'user-authentication'.
High-intent search queries and answers about installing and using this skill.
Eval-driven development is a formal evaluation framework that defines capability and regression evals before implementation to rigorously test and measure AI code performance.
You measure AI reliability by applying pass@k and pass^k metrics within an evaluation framework to quantify AI model performance and track improvements over time.
Set up regression testing for AI development by defining regression evals within an evaluation framework to ensure new features do not break existing functionality during continuous integration.
Yes, you can grade AI outputs using code-based, model-based, or human graders within an evaluation framework to assess capability and regression eval results.
No, this evaluation framework operates without external dependencies, using included scripts and references to implement eval-driven development principles for AI code sessions.
Use a formal evaluation framework for AI development when you need to benchmark model improvements, define capability evals prior to implementation, and facilitate continuous integration testing.
JamkrisAutomate eval-driven testing workflows for Gemini CLI sessions with deterministic graders.
flatrickDefine, measure, and track AI behavior with pass@k metrics.
KimjimanDefine and run capability and regression evaluations for AI code sessions.
ryasrkFormalize evaluation frameworks for AI code sessions with pass@k and pass^k metrics.
