skill-creator

Create, evaluate, and iteratively improve agent skills with benchmark-driven testing workflows.

Updated May 17, 2026
One-click install
npx skills add https://github.com/irrit-us/agent_misc --skill skill-creator-irrit-us
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/irrit-us/agent_misc/tree/main/skills/skill-creator
Command: npx skills add https://github.com/irrit-us/agent_misc --skill skill-creator-irrit-us

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and assets (resource) components.

What problem does it solve? Writing effective agent skills is hard to get right on the first try, and without structured evaluation there is no way to know whether a skill actually improves agent behavior. This Skill provides a complete create-test-review-iterate loop with quantitative benchmarking and description optimization. ## Core Features & Use Cases - Guided Skill Authoring: Interviews the user to capture intent, then drafts a SKILL.md with proper frontmatter, progressive disclosure structure, and writing patterns. - Parallel Eval Runs with Baselines: Spawns with-skill and without-skill subagent runs, grades outputs against assertions, and aggregates pass rates, timing, and token usage into benchmark reports. - Interactive Review Viewer: Generates a browser-based viewer for qualitative feedback on outputs plus a quantitative benchmark tab, feeding user comments back into the next iteration. - Description Optimization: Runs an automated train/test loop that rewrites the skill's description to improve triggering accuracy on realistic queries. - Use Case: A user wants a skill that formats weekly sales reports. This Skill drafts it, runs three realistic test prompts with and without the skill, shows the results side by side, and iterates until the outputs pass review. ## Quick Start Help me create a new skill that converts meeting transcripts into structured action-item lists, then test and refine it.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new agent skill from scratch?▼

Start by defining what the skill should do and when it should trigger, then draft a SKILL.md with name and description frontmatter plus markdown instructions. This Skill walks you through intent capture, drafting, test case creation, and iterative refinement.

How do I test whether my skill actually improves agent output?▼

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both outputs against objective assertions, then aggregate pass rates, timing, and token usage into a benchmark comparison.

How can I improve a skill description so it triggers correctly?▼

Generate 20 realistic should-trigger and should-not-trigger queries, then run the automated optimization loop. It evaluates description variants on a train split and selects the best by held-out test score to avoid overfitting.

What is the difference between with-skill and baseline eval runs?▼

With-skill runs execute the test prompt with the skill loaded, while baseline runs use no skill for new skills or the previous version for improvements. Comparing them isolates the skill's actual contribution to output quality.

When should I skip quantitative benchmarking for a skill?▼

Skip assertions and benchmarks for skills with subjective outputs like writing style or design quality, where human judgment matters more than pass/fail checks. Also skip on platforms without subagent support, such as Claude.ai.