skills.rest
NicolaeRotaru
NicolaeRotaruCOMMUNITY
Explore all skills by this author →
GitHub

ab-testing

Designs A/B tests and growth experimentation programs with statistical rigor and hypothesis frameworks.

Updated Jun 21, 2026
Search Tags:#growth#conversion optimization#hypothesis#experimentation#a/b testing#statistical significance#ice score
One-click install
npx skills add https://github.com/NicolaeRotaru/ad-mycity --skill ab-testing-nicolaerotaru
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-testing
Source: https://github.com/NicolaeRotaru/ad-mycity/tree/main/.cursor/skills/ab-testing
Command: npx skills add https://github.com/NicolaeRotaru/ad-mycity --skill ab-testing-nicolaerotaru

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? It helps you plan, run, and analyze A/B tests that produce statistically valid, actionable results instead of guesswork, and build a continuous experimentation program for growth. ## Core Features & Use Cases - Hypothesis & Test Design: Structures hypotheses, selects primary/secondary/guardrail metrics, calculates sample sizes, and defines traffic allocation. - Experimentation Program: Provides ICE prioritization, experiment velocity tracking, cadence rituals, and a playbook template for compounding wins. - MyCity Context: Adapts all recommendations to the MyCity local marketplace in Piacenza, enforcing honesty rules (no invented numbers or fake testimonials) and local channels like Facebook groups and WhatsApp. - Use Case: You want to test two versions of a landing page headline for a local shop campaign. The skill helps you write a strong hypothesis, compute the required sample size, avoid peeking early, and document the result in an experiment playbook. ## Quick Start Ask the assistant to design an A/B test for your landing page headline, including hypothesis, sample size, and success metrics.

Frequently Asked Questions about ab-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design an A/B test with statistical significance?▼

Start with a structured hypothesis, define one primary metric, and pre-calculate the required sample size before launching. Commit to running the test until that sample size is reached and use 95% confidence (p < 0.05) to call the result.

How long should I run an A/B test?▼

Run the test until you reach the pre-calculated sample size, typically 2-4 weeks. Stopping early when results look good leads to false positives, so avoid peeking and trust the pre-committed methodology.

What is ICE scoring for experiment prioritization?▼

ICE scores each hypothesis 1-10 on Impact, Confidence, and Ease, then averages the three. Run the highest-scoring experiments first and re-score monthly as context changes.

What is the difference between A/B, A/B/n, and multivariate tests?▼

A/B tests compare two versions with a single change, A/B/n tests multiple variants requiring more traffic, and multivariate tests combine several changes at once needing very high traffic. Choose based on available traffic and how many variables you must isolate.

Why did my A/B test show no significant difference?▼

Inconclusive results usually mean insufficient traffic, too small a change to detect, or an effect smaller than your minimum detectable effect. Either run longer with more traffic or design a bolder variant.

Should I use client-side or server-side A/B testing tools?▼

Client-side tools like PostHog or Optimizely are quick to implement but can cause page flicker. Server-side tools like LaunchDarkly or Split avoid flicker but require development work before rendering.

Related Skills

Explore More Marketing & Sales Skills→
aitytechaitytech
595

ab-test-setup

Designs A/B tests with hypotheses, sample size calculations, and statistical analysis plans.

Official
Intermediate
chriscarteruxchriscarterux

conversion-optimization

Design and execute A/B tests to optimize landing page and signup funnel conversions.

Community
Intermediate
PostHogPostHog
58

configuring-experiment-analytics

Configure experiment analytics workflows for exposure, metrics, and interpretation.

Official
Advanced
WcofWcof
1

pm-interview

Generate structured user interview scripts from PMContext for product discovery.

Community
Advanced