skills.rest
ensj
ensjCOMMUNITY
Explore all skills by this author →
Website𝕏@_ensjGitHub

tier-ladder

Certifies the cheapest model tier that ties the top tier for an atomic skill.

Updated Jun 20, 2026
Search Tags:#cost optimization#model evaluation#prompt testing#llm benchmarking#tier certification#bootstrap statistics
One-click install
npx skills add https://github.com/ensj/skill-eval-framework --skill tier-ladder-ensj
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tier-ladder
Source: https://github.com/ensj/skill-eval-framework/tree/main/.claude/skills/tier-ladder
Command: npx skills add https://github.com/ensj/skill-eval-framework --skill tier-ladder-ensj

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Teams running prompt-based skills on expensive models need to know whether a cheaper model tier produces statistically equivalent results, without guessing or relying on anecdotal spot checks. ## Core Features & Use Cases - Tier Sweep with Statistics: Runs the frozen winning prompt across haiku, sonnet, and opus tiers with adaptive seed batches, epsilon tie rules, and bootstrap confidence intervals. - Provenance Reconciliation: Replays relayed scores from disk via a reconcile CLI before any certification is written, blocking unverifiable results. - Durable Artifacts: Writes a validated tier-profile.json and a human-readable report.md with per-tier means, confidence intervals, and saturation diagnostics. - Use Case: After a variant sweep picks a winning prompt for a classification skill, run the tier ladder to certify it for haiku instead of opus, cutting inference cost while keeping measured quality. ## Quick Start Ask the agent to run the tier ladder for a specific skill id to certify whether it can be downgraded to a cheaper model tier.

Frequently Asked Questions about tier-ladder

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I certify a skill for a cheaper model tier?▼

Run the tier ladder for the skill id after its variant sweep has produced a winner.json. The skill runs the frozen prompt across haiku, sonnet, and opus with adaptive seeds and writes a tier-profile.json naming the cheapest tier that ties opus.

How does the tier ladder decide two model tiers tie?▼

It uses bootstrap confidence intervals with an epsilon band (default 0.05) and family-wise alpha across the tier ladder. A cheaper tier ties opus when its confidence interval against the top tier stays within epsilon.

What does the uncertified_saturated status mean?▼

It means opus saturated the eval, so the cases cannot discriminate between tiers. The report lists which cases are at ceiling; harden those eval cases and re-run the ladder before any tier can be certified.

Why does the tier ladder fail at the reconcile step?▼

The reconcile CLI replays the relayed rows from disk and exits 1 when scores cannot be reproduced, indicating a provenance break. In that case no tier-profile is written; inspect the mismatch list and rebuild dist/ if needed.

Can I change which model judges the tier comparison?▼

Yes, the judge_model argument pins the scoring agent and defaults to claude-sonnet-4-6. Pinning it explicitly prevents tier judging from silently running on the session default model.

Related Skills

Explore More Software Engineering Skills→
oathgamesoathgames

merlin-tournament

Runs adversarial tournaments to generate and evaluate improved ad copy variants.

Official
Advanced
dariopalminiodariopalminio
1

skill-creator

Orchestrate Claude skill creation, evaluation, and packaging into .skill files.

Community
Advanced
NodeJSmithNodeJSmith
1

mine.skill-eval

Automate evaluation and comparison of AI skill variants via structured grading and statistical analysis.

Community
Advanced
pycode4micropycode4micro
1

skill-manager

Manage BushServer skill package lifecycle operations and catalog consistency.

Community
Advanced