rloo

What problem does it solve?

Reinforcement learning model training often suffers from high gradient variance, especially in policy optimization with sparse or delayed rewards. RLOO uses leave-one-out baselines to stabilize training and improve sample efficiency.

Core Features & Use Cases

• RLOOTrainer and RLOOConfig for variance-reduced RLHF training
• Reward function integration using completion_ids for efficient token-based rewards
• Thinking-aware patterns and stable policy optimization for reasoning tasks

Quick Start

Run a small RLOO training session with a short dataset using RLOOTrainer and the default RLOOConfig