verl-rl-training

What problem does it solve?

This Skill provides a robust and flexible framework for training Large Language Models (LLMs) using reinforcement learning (RL) at scale, enabling advanced post-training techniques like RLHF.

Core Features & Use Cases

• Distributed Training: Supports large-scale distributed training across multiple GPUs and nodes.
• Flexible Backends: Allows swapping between different compute backends (FSDP, Megatron-LM, vLLM) and rollout engines.
• Multiple RL Algorithms: Implements various RL algorithms including PPO, GRPO, and others.
• Use Case: Fine-tune a Qwen2.5-7B model on a custom dataset using GRPO for improved math reasoning capabilities, leveraging a distributed GPU cluster.

Quick Start

Use the verl-rl-training skill to launch a GRPO training job for the Qwen/Qwen2.5-7B-Instruct model on the GSM8K dataset.