experiment-framework

What problem does it solve?

This Skill brings statistical rigor to product experiments, preventing decisions driven by noise and ensuring that feature impact is accurately measured.

Core Features & Use Cases

• Statistical Rigor: Ensures experiments have clear hypotheses, proper sample sizes, and structured analysis.
• Metric Definition: Defines primary, secondary, and guardrail metrics to understand full impact.
• Sample Size Calculation: Calculates necessary sample sizes based on baseline rates and desired MDE.
• Use Case: When launching a new feature, use this Skill to design an A/B test that statistically validates its impact on conversion rates, while ensuring no critical guardrail metrics (like error rates) degrade.

Quick Start

Use the experiment-framework skill to design an A/B test for the new checkout flow, with a primary metric of conversion rate and a guardrail metric for page load time.