experiment-metrics

What problem does it solve?

This Skill helps users select reliable and effective metrics for A/B testing and experiments, ensuring that the chosen metrics accurately reflect the impact of changes and lead to sound decision-making.

Core Features & Use Cases

• STEDII Framework Guidance: Provides a structured approach (Sensitive, Timely, Efficient, Debuggable, Interpretable, Isolated) to evaluate potential metrics.
• Metric Brainstorming: Assists in generating a list of candidate metrics for an experiment.
• Metric Scoring: Facilitates scoring metrics against the STEDII criteria.
• Primary & Guardrail Metric Selection: Guides the identification of a primary success metric and crucial guardrail metrics.
• Pre-Experiment Checks: Outlines essential validation steps like A/A tests and sample ratio checks.
• Use Case: Before launching a new feature, use this Skill to evaluate proposed metrics like "daily active users" vs. "day 7 activation rate" to ensure the chosen metric is sensitive and timely enough to provide actionable insights within the experiment window.

Quick Start

Use the experiment-metrics skill to help me brainstorm 5-10 candidate metrics for an experiment aimed at improving user onboarding completion rates.