hypothesis-testing

Generates and tests falsifiable hypotheses for ML experiments and debugging loops.

Updated Jul 8, 2026
One-click install
npx skills add https://github.com/Zhachory1/autopraxis --skill hypothesis-testing-zhachory1
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hypothesis-testing
Source: https://github.com/Zhachory1/autopraxis/tree/main/skills/hypothesis-testing
Command: npx skills add https://github.com/Zhachory1/autopraxis --skill hypothesis-testing-zhachory1

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Debugging sessions and ML experiments often drift into circular reasoning, repeated dead ends, and unfalsifiable guesses. This Skill structures the investigation as a bounded hypothesis loop with explicit confirmation and refutation criteria, a tried/rejected ledger, and stop conditions. ## Core Features & Use Cases - Falsifiable Hypothesis Generation: Produces candidate explanations with pre-declared confirming and refuting evidence, cost, and risk for each test. - Tried/Rejected Ledger: Records verdicts (confirmed, refuted, inconclusive, deferred) so ruled-out paths are never re-proposed. - Bounded Loop Control: Stops on confirmed root cause, metric hit, budget exhaustion, or diminishing returns, with escalation to a human when stuck. - Use Case: A model's validation accuracy suddenly drops. Use this Skill to list candidate causes (data leakage, distribution shift, config regression), define discriminating tests for each, run the cheapest high-information checks first, and converge on a confirmed root cause with evidence. ## Quick Start Use the hypothesis-testing skill to investigate why my model's validation accuracy dropped after the last data pipeline change and produce a hypothesis ledger with verdicts.

Frequently Asked Questions about hypothesis-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I structure a root cause analysis for a failing ML experiment?▼

List candidate hypotheses from existing evidence, define confirming and refuting evidence for each, then run low-cost high-information tests first. Record each verdict in a ledger and stop when a root cause is confirmed or the budget is exhausted.

How to avoid repeating failed experiments during debugging?▼

Maintain a tried/rejected ledger that records every tested hypothesis, its evidence, and why it was ruled out. This prevents circular reasoning and stops duplicate experiments across sessions.

What makes a hypothesis falsifiable in ML debugging?▼

A falsifiable hypothesis states a specific observation that would refute it before testing begins. If no possible evidence could disprove the claim, it must be rewritten into a testable form.

When should a hypothesis testing loop stop iterating?▼

Stop when the root cause is confirmed, the target metric is hit, the compute or time budget is exhausted, or new tests show diminishing returns. If stuck with only inconclusive results, escalate to a human with the ruled-out ledger.

Why do debugging sessions produce inconclusive results?▼

Inconclusive pileups happen when tests cannot discriminate between competing hypotheses. Fix this by designing tests whose outcomes differ across candidates, or escalate with the ruled-out ledger instead of running more ambiguous checks.