What problem does it solve? Teaching agent evaluation requires a realistic agent to test against, but building one by hand is slow and real production agents are too clean to demonstrate failure analysis. This Skill scaffolds a synthetic practice agent with intentionally seeded bugs so the failure-analyzer eval toolkit has genuine signal to work with. ## Core Features & Use Cases - Archetype-Based Scaffolding: Builds a demo agent under demo_agent/ for support, RAG, classifier, or tool-using archetypes, complete with synthetic data, 2-4 stubbed fake tools, and a starter system prompt. - Deliberate Bug Seeding: Plants 2-3 subtle, realistic bugs (unenforced policies, dropped multi-intent sub-requests, tool-call argument errors, vague deferral language) that only surface through an actual eval run. - Instructor Documentation: Logs each seeded bug, its code location, and its expected failure-analyzer category in eval/state.md under a clearly marked instructor-only section. - Use Case: An instructor teaching agent evaluation asks for a practice support agent; the Skill scaffolds one with a hidden refund-policy bug, and students then run failure-analyzer to discover and classify it. ## Quick Start Ask the agent to scaffold a demo support agent with seeded bugs under demo_agent/ so you can practice running the failure-analyzer eval toolkit on it.
