research.experiment.scaffold

Generates a uv-based PyTorch ML experiment project scaffold with Hydra configs and failing TDD tests.

Updated May 9, 2026
One-click install
npx skills add https://github.com/0h-n0/auto-research --skill research-experiment-scaffold-0h-n0
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: research.experiment.scaffold
Source: https://github.com/0h-n0/auto-research/tree/main/skills/research.experiment.scaffold
Command: npx skills add https://github.com/0h-n0/auto-research --skill research-experiment-scaffold-0h-n0

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Setting up a reproducible ML experiment project from scratch—package structure, dependency management, deterministic seeding, config files, and test skeletons—is repetitive and error-prone. This Skill automates that bootstrap so researchers can move directly from an approved experiment plan to implementation. ## Core Features & Use Cases - Project Scaffolding: Creates a uv-managed Python package under .research/<slug>/code/ with pyproject.toml, src/ modules, Hydra configs, Makefile, and test skeletons. - Reproducibility Built In: Generates a frozen RunConfig dataclass with config hashing, git revision capture, and deterministic seed setup for PyTorch. - Focus-Area Dependencies: Conditionally adds libraries like transformer_lens, lm-eval, peft/trl, or langchain based on the research focus area. - TDD Red Phase: Writes intentionally failing tests (data, model, metrics, contamination) so an ml-engineer agent can implement the Green phase. - Use Case: After approving 04_EXPERIMENT_PLAN.md in an auto-research Phase 5 workflow, invoke this Skill to produce a ready-to-implement experiment codebase with observability opt-in for W&B, MLflow, or TensorBoard. ## Quick Start Use the research.experiment.scaffold skill to generate the experiment code scaffold for the approved plan in .research/<slug>/04_EXPERIMENT_PLAN.md.

Frequently Asked Questions about research.experiment.scaffold

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scaffold a PyTorch experiment project with uv?▼

Run this Skill after approving the experiment plan; it runs uv init, writes a pyproject.toml with torch, transformers, datasets, accelerate, and hydra-core, and generates src modules, configs, and tests under .research/<slug>/code/.

What does the generated project structure include?▼

It includes pyproject.toml, src/<pkg> modules (config, data, model, train, eval, utils), tests for data/model/metrics/contamination, Hydra configs/base.yaml, a Makefile, IMPL_NOTES.md, and DATA_CARD.md.

Why do the generated tests fail right after scaffolding?▼

The failing tests are intentional TDD Red phase design. The ml-engineer agent implements data loading, model forward passes, contamination checks, and metric validation to turn them green.

Can I use W&B or MLflow tracking with the generated training script?▼

Yes, observability is opt-in via environment variables. Set WANDB_API_KEY, MLFLOW_TRACKING_URI, or TB_LOG_DIR and install the matching extra with uv sync --extra wandb; otherwise it is a silent no-op.

How does the scaffold ensure experiment reproducibility?▼

It generates a frozen RunConfig dataclass capturing seed, git revision, model, and dataset fields, computes a SHA256 config hash for run IDs, and calls set_global_determinism to fix all random seeds and enable deterministic PyTorch algorithms.