skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Configures RL post-training of LLMs using Megatron-LM training with SGLang rollout generation.

1|Updated Mar 5, 2026
Search Tags:#reinforcement learning#distributed training#sglang#rlhf#grpo#megatron-lm#llm post-training
One-click install
npx skills add https://github.com/Clay-HHK/claude-skills --skill slime-rl-training-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/Clay-HHK/claude-skills/tree/main/slime
Command: npx skills add https://github.com/Clay-HHK/claude-skills --skill slime-rl-training-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve? Setting up reinforcement learning post-training for large language models requires coordinating distributed training, high-throughput rollout generation, and data management, which is complex to wire together manually. ## Core Features & Use Cases - GRPO and PPO Training Workflows: Launch synchronous or asynchronous RL training with pre-configured model scripts for GLM, Qwen3, DeepSeek V3, and Llama 3. - Multi-Turn Agentic Training: Define custom generate functions for tool-calling and multi-step reasoning agents with proper loss masking. - Flexible Data Buffer: Implement off-policy training with custom sample filtering and prioritized replay buffers. - Use Case: Train a Qwen3-4B reasoning model with GRPO by sourcing the model script, preparing JSONL prompt data, and launching train.py with 8 GPUs for training and 8 for rollout. ## Quick Start Ask the assistant to set up a GRPO training run with slime for a Qwen3-4B model using your JSONL prompt dataset.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train an LLM with GRPO using slime?▼

Prepare a JSONL dataset with prompt and label fields, source a pre-configured model script from scripts/models/, then launch train.py with --advantage-estimator grpo, rollout batch size, and GPU allocation flags. Monitor progress through TensorBoard logs.

What is the difference between slime, verl, and miles for RL training?▼

slime provides tight Megatron-LM integration with SGLang rollout and backs GLM model training. Choose verl for flexible backend swapping, miles for enterprise stability features, or torchforge for PyTorch-native abstractions.

Which models does slime support for RL post-training?▼

slime supports GLM-4.5/4.6/4.7, Qwen3 including MoE variants, DeepSeek V3 and R1, Llama 3, Kimi K2, and Moonlight-16B. Each model has a pre-configured shell script in scripts/models/ defining architecture arguments.

Can slime do multi-turn agent training with tool calls?▼

Yes, slime supports multi-turn training through custom generate functions passed via --custom-generate-function-path. Your function handles tool execution loops and must set loss masks so tool responses are excluded from training loss.

Why does slime training OOM or hang during weight sync?▼

OOM is fixed with --recompute-activations, smaller micro-batch size, or sequence parallelism. Weight sync hangs are resolved with --colocate to avoid network transfer, or by increasing --update-weights-interval in async mode.

Does slime support asynchronous RL training?▼

Yes, train_async.py overlaps rollout and training using --async-buffer-size to buffer rollouts. Note that colocated mode is not supported with async training, so training and rollout must use separate GPUs.

Related Skills

Explore More Education & Research Skills→
castform-aicastform-ai
49

generate-data

Create train and eval JSONL datasets for Castform runs.

Official
Advanced
i-bebsii-bebsi

evaluating-llms-harness

Benchmarks LLMs across 60+ academic tasks like MMLU, GSM8K, and HumanEval.

Community
Advanced
OpenLAIROpenLAIR
1.0k

tensorboard

Visualize and analyze machine learning training runs with TensorBoard.

Official
Advanced
OpenLAIROpenLAIR
1.0k

verl-rl-training

Configure Verl backends and launch GRPO-based RL fine-tuning jobs.

Official
Advanced