skills.rest
EmaRimoldi
EmaRimoldiCOMMUNITY
Explore all skills by this author →
GitHub

setup-validation

Validate ML experiment pipelines with an 8-category pre-flight checklist.

Updated Mar 27, 2026
Search Tags:#validation#quality-assurance#reproducibility#preflight#ml-pipeline#smoke-test#baseline-check
One-click install
npx skills add https://github.com/EmaRimoldi/Claude-scholar-extended --skill setup-validation
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: setup-validation
Source: https://github.com/EmaRimoldi/Claude-scholar-extended/tree/main/skills/setup-validation
Command: npx skills add https://github.com/EmaRimoldi/Claude-scholar-extended --skill setup-validation

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Structured pre-flight checklist that verifies the entire experiment pipeline produces correct results before committing to a full sweep. Catches data bugs, model loading errors, measurement mistakes, ablation failures, and baseline mismatches — all cheaper to fix now than after 200 GPU-hours.

Core Features & Use Cases

  • Data Integrity Checks: shape checks, distribution checks, label verification, reproducibility checks, and leakage prevention.
  • Model Loading Verification: ensure non-trivial outputs, correct shapes, hooks, device alignment, and dtype conformity.
  • Measurement Correctness: validate identity/consistency, known-answer checks, numerical stability, and scaling invariants.
  • Ablation Verification: confirm zeroed components, measurable impact, restoration, and component specificity.
  • Baseline Replication: compare against published baselines or sensible trivial baselines with defined tolerance.
  • Compute Environment & Reproducibility: verify GPU availability, device consistency, CUDA version, and deterministic seeds.
  • ML-Specific Validation: gradient check, edge-case tests, minimum test coverage, and determinism checks.
  • End-to-End Smoke Test: run a complete pipeline configuration and produce a validated report plus timing data.

Quick Start

Run the full pre-flight validation checklist on the current project to generate a validation-report.md.

Frequently Asked Questions about setup-validation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run preflight validation on an ML experiment pipeline?▼

Preflight validation checks ML experiment pipelines by running a structured 8-category checklist covering data integrity, model loading, and measurement correctness to generate a validation-report.md.

Why do I need preflight checks before running a large-scale ML sweep?▼

Preflight checks catch data bugs, model loading errors, and baseline mismatches before committing compute, making issues cheaper to fix before expending 200 GPU-hours on a full run.

Can I verify ablations and baseline replication during ML pipeline validation?▼

Ablation verification confirms zeroed components and measurable impact, while baseline replication compares results against published baselines using configurable tolerances to detect regressions.

Does preflight validation test compute environment and reproducibility?▼

Compute environment and reproducibility checks verify GPU availability, device consistency, CUDA version, and deterministic seeds to ensure stable pipeline execution.

What is included in an end-to-end smoke test for an ML pipeline?▼

An end-to-end smoke test runs a complete pipeline configuration, verifying gradient checks, determinism, and edge cases while producing a validated report with timing data.

How do I validate data integrity and prevent leakage in my ML pipeline?▼

Data integrity validation performs shape checks, distribution checks, label verification, and reproducibility checks to prevent data leakage before running experiments.

Related Skills

Explore More Education & Research Skills→
dongzhuoyaodongzhuoyao
10

fail-fast-ml-engineering

Enforce explicit errors and preflight validation in ML training pipelines.

Community
Advanced
wanshuiyinwanshuiyin
14.4k

idea-discovery

Automates research idea discovery from literature surveys to validated experiment plans for ML workflows.

Community
Advanced
kaminocorpkaminocorp
1

weights-and-biases

Track ML experiments, run hyperparameter sweeps, and manage model artifacts with Weights & Biases.

Official
Intermediate
always-furtheralways-further
4

testing-assistant

Plan and write unit tests across Jest, pytest, Go, and JUnit frameworks.

Official
Advanced