run-composition

Runs typed skill compositions end-to-end and measures predicted versus actual performance deltas.

Updated Jun 20, 2026
One-click install
npx skills add https://github.com/ensj/skill-eval-framework --skill run-composition-ensj
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: run-composition
Source: https://github.com/ensj/skill-eval-framework/tree/main/.claude/skills/run-composition
Command: npx skills add https://github.com/ensj/skill-eval-framework --skill run-composition-ensj

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Verifying that a multi-step composition of AI skills actually performs as predicted is hard: individual step scores do not guarantee the wired pipeline behaves as the product-of-means model forecasts. This Skill runs a typed composition end-to-end over eval cases and seeds, scores the final artifact deterministically, and reports the delta between measured and predicted performance. ## Core Features & Use Cases - End-to-end composition execution: Resolves each step's frozen winner variant, builds the prompt/schema map, and invokes the emitted patterns/<name>.js workflow over eval cases x seeds. - Prediction validation: Computes mu_pred as the product of step winner means, compares it to the measured mean, and flags composite variance violations and biased prediction models. - Calibration gating: Refuses to run when any step winner lacks a calibrated grade, since saturated steps make the prediction vacuous. - Use Case: After freezing winners for a classify-then-synthesize pipeline, ask to measure the composition and receive a report with measured mu/sigma, delta_predicted, and a running-bias check on the prediction model. ## Quick Start Run the composition named in my request end-to-end and report the measured versus predicted performance delta.

Frequently Asked Questions about run-composition

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run a skill composition end-to-end?▼

Ensure npm run build has emitted patterns/<name>.js and per-step schemas, then invoke the Workflow tool with the composition's eval cases and seeds. The skill resolves each step's frozen winner prompt and schema, executes the pipeline, and scores the final artifact with the deterministic scorer CLI.

How is delta_predicted calculated for a composition?▼

Delta_predicted is the measured mean score minus the product of the step winner means (mu_pred). The prediction holds when the absolute delta is under 0.10, but only counts as genuine confirmation when every step skill is calibrated with a winner mean below 1.0.

Why does the composition run stop with uncalibrated steps?▼

If any step winner has a grade other than calibrated (discriminating, rejected, saturated, or underpowered), the run halts because a winner mean of 1.0 pins the product prediction at 1.0, making delta_predicted vacuous and unable to fail.

What is the difference between dispatch and inline execution for compositions?▼

Dispatch runs one subagent per step with parallel fan-out, which is right for measuring compositions and per-step scores. Inline execution in a single context is how packaged skills should ship, since it beats dispatch on quality and cost for synthesis-shaped work that fits one context.

When is a composition prediction considered biased?▼

After each run, residuals are recorded via the record-prediction CLI, which prints a running-bias line. If it reports BIASED, the product-of-means model is systematically off and should be revisited, for example by adding the deferred fanin min-input term.