skill-creator

Create, evaluate, and iteratively improve Claude skills with benchmarked test runs.

Updated Aug 17, 2026
One-click install
npx skills add https://github.com/prabaljainn/my-claude-code-setup --skill skill-creator-prabaljainn
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/prabaljainn/my-claude-code-setup/tree/main/claude/skills/skill-creator
Command: npx skills add https://github.com/prabaljainn/my-claude-code-setup --skill skill-creator-prabaljainn

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing a skill that triggers reliably and produces good outputs is hard to do by intuition alone. This Skill provides a structured workflow for drafting new skills, running test prompts with and without the skill, grading results against assertions, and iterating based on quantitative benchmarks and user feedback. ## Core Features & Use Cases - Guided Skill Authoring: Interviews the user about intent, trigger contexts, and output formats, then drafts a SKILL.md following progressive-disclosure and writing-style best practices. - Eval and Benchmark Loop: Spawns parallel with-skill and baseline runs, drafts assertions, grades outputs, and aggregates pass rates, timing, and token usage into benchmark.json with mean and standard deviation. - Description Optimization: Generates realistic should-trigger and should-not-trigger queries, reviews them in an HTML tool, and runs an automated train/test optimization loop to improve triggering accuracy. - Use Case: You want a skill that formats weekly sales spreadsheets. This Skill helps you draft it, runs three realistic test prompts with and without the skill, shows results in a browser-based review viewer, and rewrites the skill based on your feedback until the outputs pass. ## Quick Start Ask Claude to create a new skill for your workflow, for example: help me create a skill that converts meeting notes into structured action items.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Claude skill from scratch?▼

Define what the skill should do, when it should trigger, and its output format, then write a SKILL.md with name and description frontmatter plus markdown instructions. This Skill guides the interview, drafts the file, and sets up test cases to verify behavior.

How do I test whether my skill actually improves outputs?▼

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both against assertions, then aggregate pass rates, timing, and token usage into a benchmark comparing the two configurations.

How can I improve my skill description so it triggers more reliably?▼

Generate 20 realistic queries mixing should-trigger and should-not-trigger cases, then run the automated optimization loop. It evaluates descriptions on a train split, proposes improvements, and selects the best by held-out test score.

Does skill evaluation work on Claude.ai without subagents?▼

Yes, but with limitations. Without subagents you run test prompts sequentially yourself, skip baseline comparisons and quantitative benchmarking, and collect feedback inline instead of through the browser viewer.

What makes a good skill test assertion?▼

Good assertions are objectively verifiable conditions with descriptive names, such as checking that an output file exists or contains required fields. Subjective qualities like writing style are better evaluated through human review than forced assertions.