What problem does it solve? Teams shipping prompts, retrieval pipelines, or agent workflows often have no structured way to tell whether a change made the system better or worse. This Skill designs evaluation suites that give a defensible answer to "is this good enough to ship" grounded in scenarios that resemble real usage. ## Core Features & Use Cases - Scenario Design: Builds test scenarios from real failure modes and logged incidents rather than synthetic examples, covering multi-step agentic workflows and retrieval grounding. - Scoring Method Selection: Chooses among deterministic assertions, rubric-based grading, and model-graded judgment for each scenario class, with judge consistency and calibration checks. - Regression Strategy: Sets thresholds that catch real degradation without false alarms, and reports cost and latency alongside quality. - Use Case: Before shipping a new retrieval pipeline, use this Skill to produce a prioritized scenario matrix with metrics per class, a regression threshold, and an explicit statement of what still requires live or shadow validation. ## Quick Start Ask the agent to design an eval suite for your prompt or agent workflow, including failure-mode scenarios, scoring methods, and regression thresholds.
