ab-test-setup

Plan statistically valid A/B tests with sample-size calculations and documentation templates.

Updated Aug 23, 2026
One-click install
npx skills add https://github.com/mia-ppp/marketting-skillsv2 --skill ab-test-setup-mia-ppp
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/mia-ppp/marketting-skillsv2/tree/main/skills/ab-test-setup
Command: npx skills add https://github.com/mia-ppp/marketting-skillsv2 --skill ab-test-setup-mia-ppp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Design, plan, and execute statistically valid A/B tests and growth experiments to determine which approach drives meaningful improvement, while building a repeatable experimentation program that scales with your product.

Core Features & Use Cases

  • Hypothesis framework: Structure tests with clear observations, proposed changes, expected outcomes, and measurable success criteria.
  • Test type guidance: Supports A/B, A/B/n, MVT, and split URL tests across pages, features, or flows to match complexity and traffic.
  • Sample size & duration guidance: Provides structured guidance on required samples, test duration, and when to stop or extend experiments.
  • Metrics framework: Defines primary, secondary, and guardrail metrics to align experiments with business value and risk controls.
  • Variant design & testing best practices: Offers guidance on what to vary (copy, layout, CTAs, sequencing) and how to allocate traffic for reliable results.
  • Documentation & playbooks: Encourages thorough documentation of hypotheses, results, learnings, and reusable patterns for future tests.
  • Templates & cadence: Includes templates and recommended cadences to sustain a continuous growth experimentation program.

Quick Start

Identify your test context and baseline metrics, then write a hypothesis using the Because [observation], we believe [change] will cause [outcome] format and define a primary metric to measure.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size and duration for an A/B test?▼

To calculate A/B test sample size and duration, you need your baseline metrics and expected effect size. This Skill provides structured guidance on required samples, test duration, and when to stop or extend experiments for statistically valid growth experiments.

What is the best way to structure an A/B testing hypothesis?▼

The best way to structure an A/B testing hypothesis is using the format: Because [observation], we believe [change] will cause [outcome]. This framework ensures tests have clear observations, proposed changes, expected outcomes, and measurable success criteria.

Can I run multivariate and split URL tests with this experimentation framework?▼

Yes, you can run multivariate and split URL tests. The framework supports A/B, A/B/n, MVT, and split URL tests across pages, features, or flows to match your required complexity and traffic allocation.

How do I define primary, secondary, and guardrail metrics for growth experiments?▼

To define metrics for growth experiments, align primary metrics with business value and use guardrail metrics for risk controls. This Skill provides a metrics framework to structure these measurements and ensure rigorous execution.

When should I not use an A/B test and what are the limitations?▼

You should not use an A/B test when lacking sufficient traffic for sample-size requirements or clear baseline metrics. Limitations include invalid results from early stopping, insufficient variant design, and poorly defined success criteria.