skills.rest
AndurilCode
AndurilCodeOFFICIAL
Explore all skills by this author →
GitHub

context-eval

Evaluate context engineering harnesses against no-harness baselines with graded assertions.

7|2|Updated Mar 5, 2026
Search Tags:#A/B testing#agent tooling#prompt benchmarking#context evaluation#grader assertions#token cost analysis#harness optimization
One-click install
npx skills add https://github.com/AndurilCode/craftwork --skill context-eval-andurilcode
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: context-eval
Source: https://github.com/AndurilCode/craftwork/tree/main/skills/context-eval
Command: npx skills add https://github.com/AndurilCode/craftwork --skill context-eval-andurilcode

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) and scripts (resource) components.

What problem does it solve?

Context engineering often feels effective, but teams need evidence that a harness actually improves agent outcomes versus a no-harness baseline.

Core Features & Use Cases

  • A/B harness evaluation loop: runs the same realistic prompts with and without the harness, then grades outputs against explicit assertions.
  • Evidence-first grading & token-aware reporting: uses a dedicated grader protocol and generates a report (and optional HTML viewer) that summarizes benefit and cost.
  • Diagnose what’s working (and what isn’t): optionally supports blind comparison, post-hoc impact analysis, and harness optimization to prune waste and improve context precision.
  • Targets common harness goals: validates rules, instructions, guidelines, docs, retrieval pipelines, and tool setups—especially when users ask “does this context help?” or want benchmark-quality comparisons.

Quick Start

Ask the system to run context-eval for your harness and benchmark it by executing the eval prompts with and without the harness, then grading outputs using the provided assertions.

Frequently Asked Questions about context-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark an AI agent harness to prove it improves outcomes?▼

Benchmarking an agent harness requires running an A/B evaluation loop that executes the same prompts with and without the harness, grading outputs against assertions to measure outcome deltas.

What is context evaluation in prompt engineering?▼

Context evaluation is the process of measuring whether rules, instructions, and retrieval pipelines actually improve agent outcomes compared to a no-harness baseline.

How do I measure token cost benefits when adding retrieval pipelines?▼

You measure token cost benefits by running an A/B harness evaluation loop that generates a token-aware report summarizing performance benefit and cost delta.

Can I run blind comparison tests to optimize agent tooling instructions?▼

Yes, you can run blind comparison tests to diagnose what is working and optimize harness precision by pruning waste through post-hoc impact analysis.

What is the best way to validate grader assertions for agent benchmarks?▼

The best way to validate grader assertions is to apply a dedicated grader protocol within an A/B harness evaluation loop, ensuring outputs are consistently measured against explicit assertions.

Why does my context engineering harness not improve agent outcomes?▼

Your context engineering harness may not improve outcomes if added rules or retrieval pipelines lack precision, which you can diagnose using post-hoc impact analysis and blind comparison.

Related Skills

Explore More Software Engineering Skills→
Thiago-Cruz-engThiago-Cruz-eng

agent-evaluation

Design evaluation systems for AI agents with benchmarks and grader rubrics.

Community
Advanced
xu-xiangxu-xiang
1.8k

eval-harness

Define, execute, and report capability and regression evals for Claude Code sessions.

Community
Intermediate
Ikram-AlamIkram-Alam

evaluation

Evaluate agent outputs against rubrics and ground truth with multi-dimensional scoring.

Community
Basic
bugrabilgebugrabilge
10

agent-evaluation

Design benchmark test suites for evaluating LLM agent reliability.

Community
Advanced