run-experiment

Executes full-length paper reproduction runs sequentially and records structured evidence per run.

Updated Aug 14, 2026
One-click install
npx skills add https://github.com/yyccbb/PaperReproduction-Skills --skill run-experiment-yyccbb
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: run-experiment
Source: https://github.com/yyccbb/PaperReproduction-Skills/tree/main/skills/run-experiment
Command: npx skills add https://github.com/yyccbb/PaperReproduction-Skills --skill run-experiment-yyccbb

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Running full-scale machine learning experiments after validation is error-prone: commands get re-derived incorrectly, failed runs trigger ad-hoc fixes that invalidate the reproduction, and crashed sessions lose the record of completed runs. This Skill scales validated mock commands back to paper-faithful settings, executes every run sequentially, and leaves an auditable report on disk. ## Core Features & Use Cases - Manifest-driven execution: Derives full commands from the validated stage-4 commands plus stage-1's downscaling notes and sweep tables, writing the complete run manifest before launching anything. - Sequential monitored runs: Launches each run through the run-visible helper in a watchable terminal, with poll cadence scaled to the estimated wall time, and records failures as findings rather than entering fix loops. - Resume and evidence tracking: Scans prior run directories to skip already-complete runs, judges completion by log and artifact evidence (not just exit code), and writes a run.json per run plus a structured run-experiment.md report. - Use Case: After stages 1-4 of a paper reproduction pipeline finish, invoke this Skill to run a 12-run hyperparameter sweep overnight; if the session dies after run 5, re-invoking it resumes from run 6 without redoing finished work. ## Quick Start Run stage 5 of the reproduction pipeline on this repo: launch the full-length validated experiments sequentially and record everything in .paper-reproduction/run-experiment.md.

Frequently Asked Questions about run-experiment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run full-scale reproduction experiments after validating mock commands?▼

Start from the validated commands in the run-validation report, restore the duration flags listed in the scoping report's downscaling section to their paper values, and expand any sweep tuple tables into one command per row. Write the full manifest before executing, then run each command sequentially.

How do I resume an interrupted machine learning experiment sweep?▼

Scan the runs directory for existing run ids before launching. A run counts as complete only when its exit code is 0 and its log or artifacts show real work; skip those and run everything else fresh. This makes re-invoking the pipeline continue the sweep instead of restarting it.

Should I fix code or hyperparameters when a full training run fails?▼

No. Editing code, the environment, or scientifically meaningful hyperparameters after validation desynchronizes the run from what was validated. Record the failure with a diagnosis and route it back to the validation stage instead of retrying with shrunken settings.

Why is exit code 0 not enough to consider a training run complete?▼

A process can exit cleanly without doing the intended work, such as finishing suspiciously fast or skipping evaluation. Completion requires evidence in the log and artifacts: the final epoch reached, finite losses, printed metrics, and expected checkpoint files.

Can full reproduction runs execute in parallel on one GPU?▼

They should not. Full runs contend for the same GPU, and concurrent runs can slow each other or cause out-of-memory failures. Sequential execution in manifest order keeps each run's timing and memory behavior attributable and reproducible.