What problem does it solve? Running experiments without proper statistical methodology leads to false conclusions, wasted traffic, and bad product decisions. This Skill provides a rigorous framework for designing, sizing, and analyzing A/B tests so results are statistically valid and actionable. ## Core Features & Use Cases - Hypothesis & Test Design: Formulate measurable hypotheses using the if/then/because template and prioritize test elements by impact potential. - Sample Size & Duration Planning: Calculate required sample sizes from baseline conversion rate, minimum detectable effect, significance level, and power, with quick-reference tables. - Result Analysis & Decision Framework: Evaluate significance with p-values, confidence intervals, guardrail metrics, and a ship/extend/stop decision matrix. - Use Case: A growth team wants to test a new CTA button. Use this Skill to write the hypothesis, determine that 7,000 visitors per variant are needed for a 10% relative lift at 10% baseline CVR, run the test for two full weeks, and produce a structured analysis report with a ship-or-stop recommendation. ## Quick Start Ask the agent to design an A/B test for changing your checkout page headline, including the hypothesis, required sample size, and how to judge the results.
