constitutional-ai

What problem does it solve?

Constitutional AI trains models to be harmless through self-critique and AI feedback, reducing harmful outputs without human labels by using a constitution of guiding principles.

Core Features & Use Cases

• Two-phase safety alignment: supervised learning with self-critique and RL from AI feedback (RLAIF)
• Chain-of-thought critique and revision for transparent reasoning
• Potentially reduces need for human labeling in safety-critical domains
• Applicable to policy, risk assessment, and responsible AI development

Quick Start

Define a constitution of principles, generate critiques, revise outputs, and optionally train using TRL or PPO-based methods.