What problem does it solve? Flaky tests erode trust in CI pipelines and cause false failures that block releases. This Skill gives the qa-dev agent a deterministic, statistical method to compute a flakiness score per test and automatically quarantine unstable tests without losing their signal. ## Core Features & Use Cases - Flakiness score calculation: Computes failure_count / min(total_runs, 50) over a rolling window of test_run events stored in analytics/events/qa-events.jsonl, returning a float in [0.0, 1.0] or insufficient_data when fewer than 10 runs exist. - Reversible quarantine registry: Maintains analytics/qa/quarantine.json with state transitions (quarantined → monitoring → released) driven by configurable thresholds (default entry 0.20, release 0.05, 10 consecutive passing runs). - Opt-in gate: Fully no-op when qa_layer.flakiness_detection.enabled: false in factory.config.yaml, preserving backward compatibility with v2.21 pipelines. - Use Case: After a QA wave, qa-dev appends test_run events, recomputes scores, excludes quarantined tests from the pass/fail gate, and reports them under a [QUARANTINED] label while still running them in advisory mode. ## Quick Start Enable the feature by setting qa_layer.flakiness_detection.enabled to true in factory.config.yaml, then ask qa-dev to compute the flakiness score for a test and update the quarantine registry.
