What problem does it solve? Teams building agent evaluations often pick generic metrics that do not reflect what actually makes an output good or bad for their specific tasks. This Skill derives metrics from concrete good and bad examples per task, producing a grounded eval/metrics.md file instead of a copy-pasted textbook list. ## Core Features & Use Cases - Example-grounded metric derivation: Elicits 2-3 concrete good/bad output examples per task and derives metrics from the specific reasons given, never from a generic list alone. - Task-shape cross-check: Maps task types (summarization, classification, RAG, tool-using, code-gen, conversational, content-gen) to candidate metrics so nothing obvious is missed, and flags which of the 4 core metrics (relevance, faithfulness, correctness, coherence) apply. - Mandatory user confirmation gate: Presents the task-to-metric mapping for editing and pauses before writing eval/metrics.md, every time. - Use Case: After defining evaluation tasks for a customer-support agent, run this Skill to turn concrete examples of good and bad responses into a per-task metric set with definitions tied to those examples. ## Quick Start Run the metric-definition skill to derive evaluation metrics from the tasks in eval/tasks.md and write eval/metrics.md after my confirmation.
