ml-experiments

Runs ML experiments from metric definition through reproducible artifact handoff.

Updated Jul 8, 2026
One-click install
npx skills add https://github.com/Zhachory1/autopraxis --skill ml-experiments-zhachory1
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ml-experiments
Source: https://github.com/Zhachory1/autopraxis/tree/main/skills/ml-experiments
Command: npx skills add https://github.com/Zhachory1/autopraxis --skill ml-experiments-zhachory1

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? ML work often suffers from metric shopping, data leakage, untracked runs, and notebook-only results that cannot be reproduced or handed off. This Skill structures machine learning work as a scientific workflow with locked metrics, tracked experiments, and validated candidates. ## Core Features & Use Cases - Locked Metrics and Baselines: Defines primary offline metrics, online decision metrics, guardrails, and a simple baseline before any experiment results are seen. - Bounded Hypothesis Loop: Generates hypotheses with refutation criteria, trains and validates iteratively, and stops on metric success, compute caps, or diminishing returns. - Reproducible Handoff: Packages model artifacts, data lineage, tried/rejected ledgers, limitations, and deployment readiness for research or production decisions. - Use Case: A data scientist testing a new ranking model locks the offline metric and guardrails, runs tracked experiments against a baseline, validates the best candidate for leakage and fairness, and hands off a deployment-ready package. ## Quick Start Use the ml-experiments skill to frame my churn prediction problem, lock a primary metric, and run a tracked baseline experiment with a reproducible handoff.

Frequently Asked Questions about ml-experiments

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run a reproducible ML experiment workflow?▼

Start by locking the primary offline metric, online decision metric, and guardrails before viewing results. Then build a reproducible data pipeline, configure experiment tracking for params, metrics, seeds, and data lineage, and iterate through a bounded hypothesis-train-validate loop.

How to prevent data leakage in machine learning experiments?▼

Prevent leakage by using temporal splits, auditing feature sources, and running leakage checks during EDA and candidate validation. The workflow treats suspected leakage as an escalation trigger requiring statistical review before further training spend.

When should I stop iterating on ML model experiments?▼

Stop when the primary metric and guardrails meet the decision rule, when the compute budget is exhausted, or when recent iterations show diminishing returns below a configured threshold. Each stop condition produces a handoff with the best result and recommended next steps.

What should an ML experiment handoff package include?▼

A handoff package includes model artifacts, code, data lineage, recorded metrics, known limitations, deployment risks, and the next decision ask. It should be research-ready for lite runs or deployment-ready for deep production-impacting work.

Why do ML experiments fail to reproduce results?▼

Failures usually come from notebook-only work without tracked params, seeds, code versions, or data lineage. Configuring experiment tracking and maintaining a tried/rejected ledger ensures every run can be replayed and compared.