failure-analyzer

What problem does it solve? When an eval run produces failures, raw pass/fail counts and flat lists of failing examples do not explain what is actually wrong. This Skill turns eval results into an actionable diagnosis by grouping failures by root cause and surfacing regressions first. ## Core Features & Use Cases - Regression Detection: Diffs current results against the prior run and always ranks newly broken, previously-passing examples as the top category. - Root-Cause Clustering: Groups failures into categories like policy violations, dropped sub-issues, vague deferrals, tool-call errors, tone mismatch, retrieval misses, and looping behavior. - Subagent Delegation: Hands clustering to a failure-triage subagent when more than 30 examples fail, preserving context on large result sets. - Use Case: After baseline-runner writes eval/results/baseline.json with 40 failures, run this Skill to get eval/failure_report.md showing 5 regressions, 12 tool-call errors, and 8 dropped sub-issues, each with example IDs and a suggested fix direction. ## Quick Start Analyze the failures in eval/results/baseline.json and write a clustered failure report to eval/failure_report.md.