skills.rest
maximoseo
maximoseoCOMMUNITY
Explore all skills by this author →
GitHub

ab-test-setup

Design statistically valid A/B experiments with hypotheses, sample sizes, and metrics.

Updated Aug 23, 2026
Search Tags:#metrics#statistics#hypothesis#experimentation#ab-testing#sample-size
One-click install
npx skills add https://github.com/maximoseo/html-redesign-vps --skill ab-test-setup-maximoseo
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/maximoseo/html-redesign-vps/tree/main/.agents/skills/ab-test-setup
Command: npx skills add https://github.com/maximoseo/html-redesign-vps --skill ab-test-setup-maximoseo

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

A/B Test Setup helps teams design, run, and interpret statistically valid experiments to improve product metrics.

Core Features & Use Cases

  • Hypothesis-driven testing for product features, onboarding, and marketing pages
  • Clear metrics definition, sample-size planning, and guardrails
  • Documentation templates for results and stakeholder communication

Quick Start

Design a complete A/B test plan including hypothesis, metrics, variant definitions, and rollout timeline for the upcoming feature.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design a statistically valid A/B test plan for product features?▼

Designing an A/B test plan requires explicit hypotheses, predefined sample sizes, proper controls, and tracked primary and secondary metrics to ensure statistically valid product feature experiments.

What metrics do I need to track when running an A/B experiment?▼

Running an A/B experiment requires tracking primary metrics, secondary metrics, and guardrails to measure impact accurately while preventing unintended negative effects across traffic segments.

How do I calculate sample size for an A/B test on marketing pages?▼

Calculating sample size for A/B testing on marketing pages requires predefined statistical parameters to reach significant conclusions and avoid inconclusive experimentation results.

Can I use A/B testing for onboarding flows across different traffic segments?▼

A/B testing onboarding flows is fully supported across different traffic segments, requiring clear hypotheses and predefined sample sizes to measure metric improvements accurately.

What is the best way to document A/B test results for stakeholders?▼

Documenting A/B test results for stakeholders is best handled using documentation templates that capture hypotheses, variant definitions, tracked guardrails, and final experiment outcomes.

Why do I need guardrail metrics during experimentation?▼

Guardrail metrics are needed during experimentation to monitor predefined risk thresholds, preventing statistically significant wins in primary metrics from causing unintended regressions elsewhere.

Related Skills

Explore More Data & Analytics Skills→
ryzamediaryzamedia
1

ab-testing

Design and evaluate statistically rigorous A/B and multivariate experiments.

Community
Advanced
spandiosspandios

ab-test-analysis

Analyze A/B test results for statistical significance and actionable recommendations.

Community
Advanced
RebelHawk-TKRebelHawk-TK

ab-test-setup

Design A/B tests with sample size, metrics, and analysis criteria.

Community
Advanced
GuitarAlchemistGuitarAlchemist

ix-bandit

Simulate multi-armed bandit algorithms with epsilon-greedy, UCB1, and Thompson sampling.

Official
Intermediate