What problem does it solve? Evaluating an AI skill is a measurement problem, not a test-suite problem: real run traces carry no ground truth, representative failure cases are naturally scarce, and scores only mean something as deltas between skill versions. This Skill defines the standard for building trustworthy evaluation case sets and auditing agent runs so scores reflect real behavior rather than leaked answers. ## Core Features & Use Cases - Eval Case Design: Selects cases from real traces that are strong (outcome hinges on the skill), diverse (cover distinct behavior areas), and representative (replay-safe, drawn from actual runs). - Batch Evaluation Execution: Spawns one fresh agent per case via evaluate.launch, gathers results asynchronously, and verifies replays against original tool profiles, excluding degenerate runs. - Anti-Cheat Trace Auditing: Audits what agents actually did against three leak surfaces — cluster memory, web search, and completed-task results — invalidating runs that touched a leak. - Use Case: After editing a skill, re-run a reviewed case set under old and new skill text, then audit each trace to confirm score deltas come from the skill change and not from an agent reading the original run's output. ## Quick Start Use the evaluation skill to build a strong, diverse, representative case set from my recent run traces and audit the batch results for leaks.
