prompt-governance

Store prompts in a versioned registry separate from application code, either as structured files in version control or a database-backed registry. Track each version with status, promotion metadata, and an audit trail so any change can be reviewed and rolled back.

Build an eval pipeline that runs the new prompt against a golden dataset of input/expected-output pairs and compares scores to the current production version. Gate promotion on pass thresholds such as 95% exact match for classification or 0.85 LLM-as-judge score for summarization.

A golden dataset is a fixed set of input/expected-output pairs defining correct prompt behavior. It needs at least 20 examples for basic coverage and 100 or more for production confidence, covering edge cases and reviewed by a domain expert.

Assign variants by hashing user_id so each user consistently sees one variant, and log every assignment. Define the success metric before starting, run at least one week or 1,000 requests per variant, and require p<0.05 significance before declaring a winner.

This Skill is not for writing or improving individual prompts, designing RAG pipelines, or reducing LLM API costs. It applies when prompts already exist in production and need versioning, evaluation, and deployment controls.