model-ladder

Finds the cheapest model tier where an atomic skill produces a stable winner.

Updated Jun 20, 2026
One-click install
npx skills add https://github.com/ensj/skill-eval-framework --skill model-ladder-ensj
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-ladder
Source: https://github.com/ensj/skill-eval-framework/tree/main/.claude/skills/model-ladder
Command: npx skills add https://github.com/ensj/skill-eval-framework --skill model-ladder-ensj

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Choosing which model tier (haiku, sonnet, opus) to run a skill on is guesswork without measurement. This Skill automates the search for the cheapest tier that still meets a quality bar, so you avoid overpaying for expensive models or under-serving with weak ones. ## Core Features & Use Cases - Tier-by-tier climbing: Runs the variant sweep at each model tier, cheapest first, and stops as soon as a tier produces a stable winner at or above the success bar. - Strict graduation rule: A tier only counts as sufficient when the winner's score meets the bar and the outcome is "winner" or "tie" — underpowered or saturated results never graduate. - Durable artifact: Writes a validated model-profile.json recording per-tier rungs and the cheapest sufficient tier (lives_at). - Use Case: You have a classification skill and want to know if haiku is good enough. Run the ladder; if haiku's winner scores 0.92 against a 0.9 bar, the ladder stops there and records lives_at as haiku, skipping the costlier tiers. ## Quick Start Run the model ladder for my classify-text skill to find the cheapest model tier where it lives.

Frequently Asked Questions about model-ladder

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find the cheapest model for a skill?▼

Run the model ladder on the skill ID. It climbs tiers cheapest-first (haiku, sonnet, opus), running the variant sweep at each tier, and stops at the first tier whose winner meets the success bar, recording it as lives_at in model-profile.json.

How does the model ladder decide a tier is sufficient?▼

A tier is sufficient only when the sweep returns a non-null winner whose score meets or exceeds the success bar (default 0.9) and whose outcome is "winner" or "tie". Underpowered or inconclusive_saturated outcomes never graduate a tier, even above the bar.

What happens if no model tier passes the success bar?▼

The ladder runs all configured tiers and the CLI sets lives_at to null in the model profile. The per-tier rungs are still recorded, so you can inspect each tier's leaderboard and winner to decide whether to harden the eval or improve the skill.

Why did the model ladder write a 0-byte model-profile.json?▼

This happens when output is redirected straight into the destination file, which truncates it before the CLI validates. Write to a temp file first, pipe through the ladder CLI to /tmp, then move the validated result into src/skills/<skill_id>/model-profile.json.

Can I change the success bar or tier list for the ladder?▼

Yes. The defaults are success_bar 0.9, variance_ceiling 0.15, and tiers haiku, sonnet, opus, but these are parameters you can override when resolving the skill ID before climbing.