skill-creator

Create, evaluate, and iteratively improve Claude skills with benchmark testing.

1|Updated Mar 9, 2026
One-click install
npx skills add https://github.com/billiampalad/wd4 --skill skill-creator-billiampalad
Or copy as Structured Prompt for Agentโ–ผ
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/billiampalad/wd4/tree/main/.agents/skills/skill-creator
Command: npx skills add https://github.com/billiampalad/wd4 --skill skill-creator-billiampalad

SYSTEM DOCUMENTATION & REQUIREMENTS

๐Ÿ’ก This Skill includes scripts (resource) and assets (resource) components.

What problem does it solve? Building effective AI skills requires more than writing instructions โ€” you need to test whether they actually work, measure performance against baselines, and optimize trigger descriptions so the skill activates at the right time. This Skill provides a complete workflow for drafting, testing, benchmarking, and refining skills. ## Core Features & Use Cases - Skill Drafting & Interviewing: Captures user intent through structured questions, then writes SKILL.md files following progressive disclosure and writing-style best practices. - Benchmark Evaluation: Runs test cases with and without the skill in parallel, grades outputs against assertions, and aggregates pass rates, timing, and token usage into benchmark reports. - Description Optimization: Generates trigger eval queries, runs an automated optimization loop with train/test splits, and produces HTML reports showing which description triggers most accurately. - Use Case: You want to build a skill that formats meeting notes. This Skill helps you draft it, creates realistic test prompts, runs comparative evaluations, shows results in a browser-based review viewer, and iterates until the outputs meet your standards. ## Quick Start Help me create a new skill that converts meeting transcripts into structured action-item lists, then test it with a few realistic examples.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Claude skill from scratch?โ–ผ

Start by defining what the skill should do and when it should trigger, then write a SKILL.md with YAML frontmatter containing a name and description. This Skill walks you through intent capture, drafting, test case creation, and iterative evaluation.

How do I test whether my skill actually improves results?โ–ผ

Run each test prompt twice in parallel โ€” once with the skill and once without โ€” then grade outputs against objective assertions. The aggregate_benchmark.py script computes pass rates, timing, and token deltas between configurations.

How do I optimize a skill description for better triggering?โ–ผ

Generate 20 realistic should-trigger and should-not-trigger queries, review them in the HTML eval editor, then run the run_loop.py optimization script. It tests description variants against train and held-out queries and returns the best-performing description.

What is the difference between with-skill and baseline benchmark runs?โ–ผ

With-skill runs execute the test prompt with your skill loaded, while baseline runs use no skill (for new skills) or the previous version (for improvements). Comparing them isolates the skill's actual contribution to output quality.

When should I use blind comparison instead of regular benchmarking?โ–ผ

Use blind comparison when you need a rigorous judgment between two skill versions, such as verifying a rewrite is genuinely better. An independent agent compares outputs without knowing which skill produced them, reducing bias.