skills.rest
BenjaminRose805
BenjaminRose805COMMUNITY
Explore all skills by this author →
GitHub

eval-harness

Formalize AI feature validation with configurable eval structures and graders.

Updated Jan 18, 2026
Search Tags:#evaluation#agents#quality-assurance#EDD#evals#graders
One-click install
npx skills add https://github.com/BenjaminRose805/react-basecamp --skill eval-harness-benjaminrose805
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/BenjaminRose805/react-basecamp/tree/main/.claude/skills/core/eval-harness
Command: npx skills add https://github.com/BenjaminRose805/react-basecamp --skill eval-harness-benjaminrose805

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a formal evaluation framework to validate Claude Code sessions using Evaluation-Driven Development (EDD) principles, turning evaluation into a first-class artifact that guides design and implementation.

Core Features & Use Cases

  • Evaluation-driven tests for agent configurations and prompts
  • Pluggable grader system (Code-Based, Model-Based, Human) to quantify safety and correctness
  • Structured eval layout (config, cases, graders) to enable repeatable verification across features and releases

Quick Start

To get started, configure your eval directory (config.ts, cases/, graders/), then run a full suite such as pnpm eval agent-builder and review the results.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up evaluation-driven development for AI agents?▼

An eval harness validates AI features by formalizing evaluations through config, cases, and graders. It enables deterministic and model-based assessments to verify correctness and safety across prompts and agent configurations.

How do I run an evaluation suite for a Claude Code project?▼

Configure the eval directory with config.ts, cases, and graders, then run a suite like `pnpm eval agent-builder`. The harness integrates with the /implement workflow to apply graders and review assessment results.

Can I use model-based grading for AI safety checks?▼

Yes, you can use model-based grading for AI safety checks. The framework supports a pluggable grader system including Code-Based, Model-Based, and Human graders to quantify safety and correctness across features.

Does this evaluation framework work without external dependencies?▼

Yes, the evaluation framework works without external dependencies. It uses a configurable internal structure of config, cases, and graders to enable repeatable verification directly within Claude Code projects.

What is the best way to structure tests for AI feature validation?▼

The best way to structure AI feature validation tests is using a formal eval layout separating config, cases, and graders. This enables repeatable verification and integrates testing pipelines with implementation workflows.

Related Skills

Explore More Software Engineering Skills→
AndurilCodeAndurilCode
7

context-eval

Evaluate context engineering harnesses against no-harness baselines with graded assertions.

Official
Advanced
RHEcosystemAppEngRHEcosystemAppEng
10

run-evals

Automate execution and grading of AI skill evaluations for CI pipelines.

Official
Advanced
Clean1inesClean1ines

eval-harness

Evaluate Claude Code sessions with configurable graders and run pipelines.

Community
Advanced
as4584as4584
2

llm-evaluation

Evaluate LLM applications with automated metrics and human feedback.

Community
Advanced