skills.rest
hanasho744
hanasho744COMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Guides LLM post-training with reinforcement learning using Megatron-LM and SGLang.

Updated Mar 23, 2026
Search Tags:#reinforcement learning#distributed training#sglang#rlhf#grpo#megatron-lm#llm post-training
One-click install
npx skills add https://github.com/hanasho744/codex --skill slime-rl-training-hanasho744
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/hanasho744/codex/tree/main/.agents/skills/orchestra-slime
Command: npx skills add https://github.com/hanasho744/codex --skill slime-rl-training-hanasho744

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve? Setting up reinforcement learning post-training for large language models requires coordinating distributed training, high-throughput rollout generation, and custom reward logic, which is complex to configure correctly. ## Core Features & Use Cases - GRPO and PPO Training Workflows: Step-by-step guidance for launching RL training runs with Megatron-LM parallelism and SGLang-based rollout generation. - Multi-Turn Agentic Training: Instructions for implementing custom generate functions with tool calling and loss masking for agent training. - Troubleshooting Reference: Solutions for common failures including SGLang crashes, weight sync timeouts, OOM errors, and reward collapse. - Use Case: Train a Qwen3-4B model with GRPO on math reasoning data by sourcing a pre-configured model script, preparing JSONL prompt data, and launching train.py with the documented argument set. ## Quick Start Ask the assistant to help you launch a GRPO training run with slime for your model and dataset, and it will walk you through data preparation, model configuration, and the training command.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train an LLM with GRPO using slime?▼

Prepare a JSONL dataset with prompt and label fields, source a pre-configured model script from scripts/models/, then launch train.py with --advantage-estimator grpo, rollout batch size, and samples per prompt. Ensure rollout_batch_size times n_samples_per_prompt equals global_batch_size times num_steps_per_rollout.

What models does slime support for RL training?▼

slime supports GLM-4.5 through GLM-4.7, Qwen3 including MoE variants, DeepSeek V3 and R1, Llama 3, Kimi K2, and Moonlight-16B. Each model family has pre-configured shell scripts in scripts/models/ defining architecture and checkpoint arguments.

slime vs verl vs torchforge for RL post-training?▼

Choose slime for Megatron-LM native training with SGLang inference and GLM model support. Use verl for flexible backend swapping, torchforge for PyTorch-native abstractions, or miles for enterprise stability features.

Can I use async training with colocated GPUs in slime?▼

No, colocated mode is not supported with train_async.py. For async training, allocate separate GPUs for training and rollout, and tune --async-buffer-size and --update-weights-interval to control weight staleness.

Why does slime training OOM during the backward pass?▼

CUDA OOM during training typically comes from large batch sizes or long sequences. Enable --recompute-activations for gradient checkpointing, reduce --micro-batch-size to 1, or turn on --sequence-parallel to distribute memory.

How do I train a multi-turn agent with tool calling in slime?▼

Write a custom async generate function that loops over turns, executes tool calls, and appends results to the conversation, then pass it via --custom-generate-function-path. Set the sample loss_mask so tool responses are excluded from loss computation.

Related Skills

Explore More Software Engineering Skills→
Miosa-osaMiosa-osa
226

synthetic-data

Generate labeled synthetic test inputs via dimension-based combinatorics for ML eval pipelines.

Official
Advanced
i-bebsii-bebsi

evaluating-llms-harness

Benchmarks LLMs across 60+ academic tasks like MMLU, GSM8K, and HumanEval.

Community
Advanced
castform-aicastform-ai
49

verify-environment

Validate Castform training environments by interpreting scorecards and reward variance.

Official
Advanced
tadod12tadod12

verl-rl-training

Configure scalable Verl RL training workflows for LLMs across vLLM and Megatron backends.

Community
Advanced