run-validation

Runs mock experiment commands and fixes code errors in a git-tracked loop until each passes.

Updated Aug 14, 2026
One-click install
npx skills add https://github.com/yyccbb/PaperReproduction-Skills --skill run-validation-yyccbb
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: run-validation
Source: https://github.com/yyccbb/PaperReproduction-Skills/tree/main/skills/run-validation
Command: npx skills add https://github.com/yyccbb/PaperReproduction-Skills --skill run-validation-yyccbb

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Reproducing a machine learning paper requires proving that every planned experiment actually executes end-to-end before committing to expensive full-length runs. This Skill automates stage 4 of a paper-reproduction pipeline: it executes the mock commands written by earlier stages, classifies each failure, fixes code bugs one git commit at a time, and delegates environment errors to the environment-setup skill. ## Core Features & Use Cases - Error-driven fix loop: Runs each mock command in the prepared conda env, classifies failures as environment, code, or asset errors, and iterates until the run passes or a stop rule fires. - Git-tracked repairs: Creates a run-validation branch where every logical fix is a separate, revertable commit with a structured message tracing back to the error. - Honest validation: Requires exit code 0 plus evidence of real work (finite losses, logged metrics, output artifacts) before declaring a mock run passed. - Use Case: After scoping, downloading, and environment setup are done for a cloned paper repo, invoke this Skill to validate all planned experiments, producing a run-validation.md report with validated commands, a fix log, and notes for the full runs. ## Quick Start Run stage 4 of the reproduction pipeline on this repo: execute the mock commands from the scoping report and fix whatever breaks until they pass, tracking every fix in git.

Frequently Asked Questions about run-validation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate that a paper's experiments actually run before launching full training?▼

Run each experiment's downscaled mock command in the prepared conda environment and fix failures iteratively until it exits 0 with real evidence of work, such as logged training steps and finite losses. This Skill automates that loop and records every fix as a separate git commit.

How should code fixes be tracked when reproducing a research paper?▼

Create a dedicated git branch and commit each logical fix separately, immediately after applying it and before rerunning. One commit may touch several files for the same fix, but unrelated fixes must never be bundled so any wrong fix can be reverted alone.

What should I do when a reproduction run fails with ModuleNotFoundError?▼

Classify it as an environment error and delegate it to the environment-setup skill in repair mode with the env name and full traceback, rather than pip-installing into the env yourself. Casual installs bypass lockfile discipline and desynchronize the recorded environment.

Can I shrink batch sizes or skip failing code to make a mock run pass?▼

No. Changing scientifically meaningful hyperparameters or wrapping failures in try/except invalidates the experiment. Record issues like OOM as findings for the full run instead of papering over them.

When should I stop trying to fix a failing experiment?▼

Stop when the same error recurs unchanged twice after a targeted fix, after roughly ten fix-rerun cycles without a pass, or when the required fix is scientific rather than mechanical. Mark the experiment FAILED with its best diagnosis and continue with the rest.