reproducibility-first

Require reproducible evidence before editing code for CI failures or flaky tests.

2|Updated Apr 20, 2026
One-click install
npx skills add https://github.com/veegee82/loss-driven-development --skill reproducibility-first
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: reproducibility-first
Source: https://github.com/veegee82/loss-driven-development/tree/main/skills/reproducibility-first
Command: npx skills add https://github.com/veegee82/loss-driven-development --skill reproducibility-first

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Acting on a single failure point can mislead decisions and lead to unnecessary edits. This skill enforces reproducible evidence before any downstream diagnosis or code change.

Core Features & Use Cases

  • Enforces the rule that a single observation is not a gradient, requiring reproduction or a proven unambiguous signal.
  • Branch A — Reproduce before editing: perform at least two additional runs in a matching environment to determine if the issue is transient, flaky, or deterministic.
  • Branch B — Unambiguous-signal shortcut: when the log clearly names a deterministic cause and matches a known contract violation, proceed to the appropriate fix path.
  • Integrates with downstream skills (root-cause-by-layer, loss-backprop-lens) to ensure a structured, safe gradient-based workflow.

Quick Start

Reproduce the observation by running the case again in an environment as close as possible to the original.

Frequently Asked Questions about reproducibility-first

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I stop fixing flaky tests and CI failures based on a single observation?▼

To stop chasing flaky tests and CI failures from single observations, enforce reproducible evidence before editing code. Require at least two additional reproductions in a matching environment to determine if the issue is transient or deterministic.

When debugging surprising logs, when can I skip reproducing the failure before applying a fix?▼

When debugging surprising logs, you can skip reproduction if the log clearly names a deterministic cause and matches a known contract violation. This unambiguous-signal shortcut allows proceeding directly to the appropriate fix path.

What is the best way to validate root-cause analysis for CI failures before making code edits?▼

The best way to validate root-cause analysis for CI failures is to perform at least two additional runs in an environment matching the original. This reproduction-first approach prevents misleading gradient-based fixes from transient or flaky test behavior.

How does log analysis fit into a reproducibility-first debugging workflow?▼

Log analysis in a reproducibility-first debugging workflow identifies unambiguous signals that might bypass the need for multiple reproductions. If the log clearly names a deterministic cause matching a known contract violation, it validates the failure without additional runs.

Can I use this approach for intermittent test failures in my continuous integration pipeline?▼

Yes, you can use this approach for intermittent test failures in continuous integration pipelines. By requiring at least two additional reproductions in a matching environment, it distinguishes between transient flaky tests and deterministic failures before any edits.