What problem does it solve? Prompt changes can silently degrade output quality, and without a recorded baseline there is no way to detect regression. This Skill runs a deliberate, cost-aware evaluation harness for a specific prompt version, records quality metrics and cost, and compares them against the previous version so regressions are caught before they ship. ## Core Features & Use Cases - Tiered oracle evaluation: Runs structural assertions (schema, token caps, banned strings), reference-based similarity checks, and rubric-scored LLM judge evaluations with n=3 median sampling and variance flagging. - Baseline comparison and verdicts: Aggregates pass rates, judge scores, latency, and cost, then issues a pass, regression, or below_threshold verdict against the recorded baseline, with regression-origin cases always treated as blockers. - Persistent run records: Writes results to .agent/state/evals/, updates the eval spec's baselines table, the prompt spec's version history, and the registry metadata. - Use Case: After publishing prompt version summarize@1.2.0, invoke /eval summarize to run its golden-set cases, confirm the pass rate and judge scores did not regress versus 1.1.0, and record the run as the new baseline. ## Quick Start Ask the AI to run /eval for a prompt name such as "summarize" to execute its golden-set evaluation suite and report the verdict versus the previous baseline.
