skill-creator

Create, evaluate, and iteratively improve Claude skills with benchmarked test runs.

9|Updated Feb 16, 2026
One-click install
npx skills add https://github.com/mathisdrn/orca --skill skill-creator-mathisdrn
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/mathisdrn/orca/tree/main/.agents/skills/skill-creator
Command: npx skills add https://github.com/mathisdrn/orca --skill skill-creator-mathisdrn

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing an effective skill is hard to get right on the first try — descriptions may fail to trigger, instructions may be ambiguous, and there is no built-in way to know whether a skill actually improves results. This Skill provides a structured workflow for drafting skills, running test cases with and without the skill, grading outputs against assertions, and iterating based on quantitative benchmarks and user feedback. ## Core Features & Use Cases - Guided skill authoring: Captures intent through interview questions, then writes SKILL.md with proper frontmatter, progressive disclosure structure, and writing-style guidance. - Evaluation harness: Spawns parallel with-skill and baseline runs, drafts verifiable assertions, grades outputs, and aggregates pass rates, timing, and token usage into benchmark.json via scripts/aggregate_benchmark.py. - Description optimization: Generates trigger/no-trigger eval queries, reviews them in an HTML template, and runs an automated loop to improve the skill description's triggering accuracy. - Use Case: You want a skill that formats weekly sales reports. This Skill helps you draft it, creates three realistic test prompts, runs them with and without the skill, shows results in a review viewer, and iterates until the outputs pass your assertions. ## Quick Start Help me create a new skill that converts meeting transcripts into structured action-item lists, then test and improve it.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Claude skill from scratch?▼

Start by defining what the skill should do, when it should trigger, and its expected output format. Then write a SKILL.md with name and description frontmatter plus markdown instructions, and optionally add scripts, references, or assets directories for bundled resources.

How do I test whether my skill actually improves results?▼

Run each test prompt twice in parallel — once with the skill and once without as a baseline. Grade both outputs against objective assertions, then aggregate pass rates, timing, and token usage into a benchmark to compare configurations.

How can I improve a skill description so it triggers correctly?▼

Generate 20 realistic eval queries split between should-trigger and should-not-trigger cases, then run the optimization loop. It evaluates description variants on train and held-out test queries and selects the best-scoring description.

What makes a good skill evaluation assertion?▼

Good assertions are objectively verifiable with descriptive names, such as checking that an output file exists or contains specific fields. Subjective qualities like writing style are better evaluated through human review than forced into assertions.

When should a skill include scripts versus inline instructions?▼

Bundle a script when test runs show subagents repeatedly writing the same helper code, or when a task is deterministic and repetitive. Scripts execute without loading into context, saving tokens on every invocation.

Does skill evaluation work without subagents, like on Claude.ai?▼

Yes, but with limitations. Without subagents you run test cases sequentially yourself, skip baseline comparisons and quantitative benchmarking, and rely on inline qualitative feedback instead of the browser-based review viewer.