ab-hypothesis-framing

To frame a falsifiable A/B test hypothesis, define a concrete subject, propose specific changes, and pre-register primary, secondary, and guardrail metrics. This ensures your experiment design has an explicit mechanism and reduces bias before execution.

The best way to calculate sample size and minimum detectable effect (MDE) is to plan them during experiment design. Estimating MDE and sample size upfront ensures your A/B test has sufficient statistical power to detect meaningful changes.

Prevent p-hacking and peeking by establishing explicit peeking controls and a risk register before running experiments. Applying a pre-committed decision rule ensures your statistical analysis remains unbiased throughout the A/B test duration.

Yes, you can plan A/B experiments with multiple variants. The framing process explicitly supports defining primary, secondary, and guardrail metrics alongside duration planning to ensure robust experiment design across all tested variations.

You need to define a pre-committed decision rule for experiment design whenever running A/B tests. Establishing this rule before execution, alongside duration estimation and MDE planning, prevents biased outcomes and ensures rigorous statistical validation.