verl

What problem does it solve?

Train RLHF LLMs at scale by coordinating rollout generation, policy updates, and reward signals in a unified framework.

Core Features & Use Cases

• Supports PPO, GRPO, DAPO, RLOO, and REINFORCE++ with stable training loops.
• Integrates vLLM/SGLang for fast rollout generation and FSDP or Megatron-LM for scalable training.
• Includes an SFT trainer and end-to-end RLHF pipelines for production-grade experiments.
• Provides multi-GPU scaling, reward-model integration, checkpointing, and monitoring via WandB.

Quick Start

Configure your training in a YAML file, point verl at your data, and run the trainer to start RLHF fine-tuning of your model.