What problem does it solve? Research experiments often suffer from moving goalposts, cherry-picked metrics, and uncontrolled comparisons that invalidate results. This Skill enforces a disciplined experiment planning process that fixes the metric, controls, budget, and decision rule before any training run launches. ## Core Features & Use Cases - Screening Run Planning: Design cheap small-scale training screens that compare the active candidate against a frozen baseline while changing only the mapped adaptation, with precommitted pass thresholds and valid-failure routes. - Confirmation Run Planning: Define full-budget confirmation runs with frozen code and hyperparameters, predetermined seeds, strong comparators, and precommitted aggregation and promotion thresholds. - Validity Protection: Keep held-out labels out of candidate selection, forbid protocol changes after results, and record backend, runtime, launch command, and run id bookkeeping in research/EXPERIMENT_PLAN.md. - Use Case: Before launching a training run to test a new adaptation hypothesis, use this Skill to write a plan that states which claim the run tests, what success means, and what a valid failure looks like, so the result can support or reject the hypothesis without ambiguity. ## Quick Start Ask the agent to plan a screening experiment for the active candidate and write the plan to research/EXPERIMENT_PLAN.md before launching training.
