pi-skill-creator

Create, evaluate, and iteratively improve Agent Skills for pi, opencode, and codex.

2|Updated Sep 14, 2026
One-click install
npx skills add https://github.com/kwanpham2195/agent-stuff --skill pi-skill-creator-kwanpham2195
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: pi-skill-creator
Source: https://github.com/kwanpham2195/agent-stuff/tree/main/pi/skills/pi-skill-creator
Command: npx skills add https://github.com/kwanpham2195/agent-stuff --skill pi-skill-creator-kwanpham2195

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Writing an effective Agent Skill is hard to get right on the first try: descriptions may fail to trigger on relevant queries or over-trigger on unrelated ones, and there is no built-in way to measure whether a skill actually improves agent output. This Skill provides a complete draft-test-review-improve loop with quantitative benchmarks. ## Core Features & Use Cases - Skill Authoring Guidance: Step-by-step workflow for capturing intent, writing SKILL.md frontmatter and instructions, and structuring scripts, references, and assets. - A/B Evaluation Harness: Runs test prompts with and without the skill via subagents, grades outputs against expectations, and aggregates pass rates, timing, and token usage into benchmark.json and an HTML review report. - Description Optimization Loop: Iteratively rewrites the skill description using train/test trigger evals to maximize triggering accuracy, with blinded history to avoid overfitting. - Packaging: Bundles a validated skill folder into a distributable .skill zip file. - Use Case: You wrote a skill for generating release notes but the agent rarely uses it. Run the trigger eval loop to test 20 realistic queries, let the optimizer rewrite the description, and verify the improved triggering rate in the HTML report. ## Quick Start Ask the agent to create a new skill for your task, then run the evals and review the benchmark report to iteratively improve it.

Frequently Asked Questions about pi-skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Agent Skill for pi or opencode?▼

Create a directory containing a SKILL.md file with YAML frontmatter defining a name and description, followed by Markdown instructions. Optionally add scripts, references, and assets directories, then place the folder in ~/.pi/agent/skills/ or ~/.agents/skills/.

How do I test whether my skill description triggers correctly?▼

Create a trigger eval set of about 20 queries mixing should-trigger and should-not-trigger cases, then run the run_loop script with your agent, model, and skill path. It evaluates triggering rates and iteratively rewrites the description to improve accuracy.

What agents does this skill support for running evals?▼

The scripts support pi, opencode, and codex through a shared agent configuration in scripts/__init__.py. Each agent is invoked via its CLI in headless mode, and the agent can be selected with the --agent flag or the AGENT environment variable.

How does the with-skill versus without-skill benchmark work?▼

The eval runner spawns subagents for each test prompt in two configurations: one with the skill injected and one baseline without it. Grading results are aggregated into benchmark.json comparing pass rates, execution time, and token usage, with an HTML report for review.

What are the limitations of description optimization?▼

The optimizer can overfit to specific test prompts if the eval set is too small or narrow. Use a holdout test split, write near-miss should-not-trigger queries, and generalize fixes to underlying instruction problems rather than individual failing cases.