skills.rest
daemon-blockint-tech
daemon-blockint-techCOMMUNITY
Explore all skills by this author →
GitHub

ab-testing-engineer

Frame and govern A/B experiments with hypothesis, metrics, and randomization.

7|1|Updated May 19, 2026
Search Tags:#hypothesis#experimentation#ab-testing#sample-size#power-analysis#srm#experiment-registry
One-click install
npx skills add https://github.com/daemon-blockint-tech/Agentic-Enteprises-Skill --skill ab-testing-engineer
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-testing-engineer
Source: https://github.com/daemon-blockint-tech/Agentic-Enteprises-Skill/tree/main/ab-testing-engineer
Command: npx skills add https://github.com/daemon-blockint-tech/Agentic-Enteprises-Skill --skill ab-testing-engineer

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Frames experimentation engineering to turn product or growth questions into testable, decision-ready experiments, ensuring designs are statistically sound, instrumented correctly, and interpretable.

Core Features & Use Cases

  • Hypothesis framing and pre-registered metrics to guide what to test and how decisions will be made.
  • Support for A/B, A/B/n, and multivariate (MVT) designs with clear randomization units and SRM considerations.
  • Instrumentation guidance, analysis planning, and governance through an experiment registry to track ownership and lifecycle.

Quick Start

Plan an A/B test by defining the hypothesis, metrics, randomization, sample size, and governance steps for a new feature.

Frequently Asked Questions about ab-testing-engineer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I frame a hypothesis and define metrics for an A/B test?▼

A/B test hypothesis framing requires defining testable assumptions and pre-registering metric contracts. This process guides what to test and how decisions will be made, ensuring your experimentation is decision-ready and statistically sound from the start.

What is the best way to calculate sample size and run power analysis for experimentation?▼

Power analysis determines the minimum sample size needed for your A/B experiment to detect a meaningful effect. Proper calculations ensure your test has sufficient statistical power to validate product changes without wasting resources.

How do I check for SRM (Sample Ratio Mismatch) when running A/B tests?▼

Checking for SRM involves verifying that randomization correctly distributes users across experimental variants. Properly configured instrumentation and clear randomization units prevent sample ratio mismatch and ensure valid experiment analysis.

Can I use this approach for multivariate testing (MVT) and A/B/n designs?▼

Yes, robust experimentation supports A/B, A/B/n, and multivariate (MVT) designs. These experimental designs accommodate clear randomization units and SRM considerations to validate product, growth, and marketing initiatives effectively.

How do I set up an experiment registry for governance and lifecycle tracking?▼

An experiment registry provides governance hygiene by tracking ownership and the experiment lifecycle. It incorporates pre-registration, stopping rules, and governance steps to maintain rigorous oversight across all product and growth testing initiatives.

Related Skills

Explore More Product & Management Skills→
TheCraigHewittTheCraigHewitt
7.3k

analytics-tracking

Design and implement GA4 and GTM analytics tracking with event taxonomy and privacy compliance.

Community
Intermediate
w95w95
145

statistical-analysis

Apply descriptive statistics, trend analysis, outlier detection, and hypothesis testing to datasets.

Community
Intermediate
Arnutt-NArnutt-N

feature-flags

Manage feature flags with targeted rollouts and variants across environments.

Community
Advanced
GoldenZeroGoldenZero
1

care-program-effectiveness

Measure clinical, utilization, and financial impact of care management programs.

Community
Advanced