skill-creator

Create, evaluate, and iteratively improve Claude skills with benchmarked test runs.

Updated Jul 4, 2024
One-click install
npx skills add https://github.com/hareki/dotfiles --skill skill-creator-hareki
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/hareki/dotfiles/tree/main/agents/.agents/skills/skill-creator
Command: npx skills add https://github.com/hareki/dotfiles --skill skill-creator-hareki

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing an effective skill is hard to get right on the first try, and there is no built-in way to know whether a skill actually improves Claude's output or triggers when it should. This Skill provides a structured draft-test-review-improve loop with quantitative benchmarking so skill quality is measured rather than guessed. ## Core Features & Use Cases - Guided skill authoring: Interview-driven capture of intent, trigger contexts, and output formats, then generation of a well-structured SKILL.md following progressive disclosure and writing-style best practices. - Evaluation harness: Runs test prompts with and without the skill in parallel subagents, grades outputs against assertions, and aggregates pass rates, timing, and token usage into benchmark.json with mean and standard deviation. - Description optimization: Generates realistic should-trigger and should-not-trigger queries, then runs an automated train/test optimization loop to improve the skill's triggering accuracy. - Use Case: You want a skill that formats weekly sales reports. This Skill helps you draft it, runs three realistic test prompts with and without the skill, opens a browser viewer for side-by-side review, and iterates until the outputs pass your assertions. ## Quick Start Help me create a new skill that converts meeting transcripts into structured action-item lists, then test it with a few realistic prompts.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Claude skill from scratch?▼

Start by defining what the skill should do, when it should trigger, and its expected output format. Then write a SKILL.md with name and description frontmatter, draft 2-3 realistic test prompts, and iterate based on evaluation results.

How do I test whether my skill actually improves Claude's output?▼

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both outputs against objective assertions, then compare pass rates, timing, and token usage in the aggregated benchmark.

How can I improve my skill's description so it triggers correctly?▼

Generate 20 realistic queries split between should-trigger and should-not-trigger cases, then run the optimization loop. It evaluates description variants on a train set and selects the best performer on a held-out test set to avoid overfitting.

What makes a good skill description for triggering?▼

A good description states both what the skill does and specific contexts when to use it, including phrasings users might actually type. Slightly assertive wording helps because Claude tends to undertrigger skills by default.

When should a skill include scripts, references, or assets?▼

Add scripts for deterministic repetitive tasks that test runs keep reimplementing, references for detailed documentation loaded on demand, and assets for templates or files used in outputs. Keep SKILL.md itself under roughly 500 lines.

Does skill evaluation work on Claude.ai without subagents?▼

On Claude.ai you run test prompts sequentially yourself instead of spawning parallel subagents, and skip baseline comparisons and quantitative benchmarking. Review happens inline in the conversation rather than through the browser viewer.