pre-registering-eval-study

Preregistration locks your study's hypothesis, primary outcome, stopping rule, and falsification criteria before data collection. This prevents p-hacking and HARKing by requiring a versioned document with a recorded commit hash and a power-aware analysis plan.

Yes, preregistration supports confirmatory study workflows for LLM evals, A/B tests, and model comparisons. You must specify a directional, falsifiable primary hypothesis and an operationalized outcome before observing any evaluation data.

Hypothesis locking transforms vague research ideas into a directional, falsifiable primary hypothesis with an operationalized outcome. It fixes your statistical test, minimum effect of interest, and falsification criterion in advance to ensure defensible study design.

To set up a power-aware analysis plan, specify your primary outcome, fixed effect threshold, minimum effect of interest, and stopping rule. This plan must be recorded in a versioned preregistration document with a commit hash before any data is observed.

You must lock stopping rules and falsification criteria before any data is observed to prevent optional stopping and outcome switching. This is required for clinical, observational, or experimental studies that will support a confirmatory claim.

Preregistration for observational studies requires a fixed effect threshold and pre-specified power-aware analysis plan. You cannot use it for exploratory analyses without a falsifiable primary hypothesis, an operationalized outcome, and a recorded commit hash.