gen-eval-scenario

Create gen-eval scenario YAML files with schema validation.

4|1|Updated Jan 16, 2026
One-click install
npx skills add https://github.com/jankneumann/agentic-coding-tools --skill gen-eval-scenario
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gen-eval-scenario
Source: https://github.com/jankneumann/agentic-coding-tools/tree/main/.agents/skills/gen-eval-scenario
Command: npx skills add https://github.com/jankneumann/agentic-coding-tools --skill gen-eval-scenario

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml, and includes scripts (resource) components.

What problem does it solve?

Manually writing gen-eval scenario YAML files is time‑consuming, error‑prone, and hard to keep consistent across testing contexts.

Core Features & Use Cases

  • Descriptor Auto‑Detection: Finds the appropriate interface descriptor without user input.
  • Coverage Analysis: Shows existing scenario coverage and suggests under‑tested categories.
  • Interactive Prompting: Guides the user through scenario intent, category, interfaces, and behavior.
  • Validated Output: Generates a schema‑compliant YAML file and runs a Pydantic validation step.

Use case: A developer wants to add a new failure scenario for the authentication boundary; this skill walks them through selecting transports, defining steps, and producing a ready‑to‑run YAML file.

Quick Start

Run /gen-eval-scenario --category auth-boundary --type failure to create a new scenario file.

Frequently Asked Questions about gen-eval-scenario

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate gen-eval scenario YAML files for interface testing?▼

You can generate gen-eval scenario YAML files interactively by specifying scenario intent, category, interfaces, and behavior; the tool then auto-detects interface descriptors and outputs a schema-validated YAML file ready for evaluation.

What is the best way to ensure my testing scenario YAML files pass schema validation?▼

The best way to ensure testing scenario YAML files pass schema validation is to use an interactive scenario generator that runs a Pydantic validation step automatically, verifying descriptor analysis and schema compliance before outputting the final file.

How do I find under-tested categories in my existing testing scenario coverage?▼

To find under-tested categories in your existing testing scenario coverage, use a coverage analysis tool that examines current scenario definitions, suggests missing categories, and auto-detects appropriate interface descriptors across transports.

Can I create failure scenarios for authentication boundaries without manually writing YAML?▼

Yes, you can create failure scenarios for authentication boundaries without manually writing YAML by using interactive scenario generation, which walks you through selecting transports and defining steps to produce a ready-to-run, schema-compliant YAML file.

Do I need pyyaml installed to interactively craft scenario descriptors?▼

Yes, you need pyyaml installed to interactively craft scenario descriptors, as it is the required dependency for parsing and generating the validated YAML output files used in gen-eval testing workflows.