skills.rest
80portisfound
80portisfoundCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

slime-rl-training

Guides LLM post-training with reinforcement learning using Megatron-LM and SGLang.

Updated May 3, 2026
Search Tags:#reinforcement learning#distributed training#glm#sglang#grpo#megatron-lm#llm post-training
One-click install
npx skills add https://github.com/80portisfound/vibe-learning --skill slime-rl-training-80portisfound
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/80portisfound/vibe-learning/tree/main/packages/hermes/optional-skills/mlops/slime
Command: npx skills add https://github.com/80portisfound/vibe-learning --skill slime-rl-training-80portisfound

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve? Setting up reinforcement learning post-training for large language models requires coordinating distributed training, high-throughput rollout generation, and custom reward logic, which is complex to configure correctly. ## Core Features & Use Cases - GRPO and Async Training Workflows: Step-by-step guidance for standard GRPO training, asynchronous rollout-training overlap, and multi-turn agentic training with tool calling. - Configuration Reference: Covers Megatron arguments, SGLang-prefixed arguments, and slime-specific parameters including the batch size constraint formula. - Troubleshooting Playbooks: Solutions for SGLang engine crashes, weight sync timeouts, CUDA OOM, reward collapse, and checkpoint loading failures. - Use Case: You want to train a Qwen3-4B model with GRPO on math reasoning data. This Skill walks you through data formatting, model script selection, launch commands, and monitoring. ## Quick Start Ask the assistant to help you launch a GRPO training run with slime for your model and dataset, specifying your GPU count and data path.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train an LLM with GRPO using slime?▼

Prepare a JSONL dataset with prompt and label fields, source a pre-configured model script from scripts/models/, then launch train.py with --advantage-estimator grpo, rollout batch size, samples per prompt, and your model arguments. Monitor progress via TensorBoard.

slime vs verl vs miles for RL post-training?▼

Choose slime for Megatron-LM native training with SGLang inference and GLM model support. Use miles for enterprise stability features, verl for flexible backend swapping, or torchforge for PyTorch-native abstractions.

Which models does slime support for RL training?▼

slime supports GLM-4.5/4.6/4.7, Qwen3 including MoE variants, DeepSeek V3 and R1, Llama 3, Kimi K2, and Moonlight-16B. Each model has pre-configured scripts in the scripts/models/ directory.

Why does SGLang crash during slime training?▼

SGLang engine crashes mid-training usually stem from memory pressure. Enable --use-fault-tolerance, raise --sglang-mem-fraction-static to 0.85, reduce rollout batch size, or disable CUDA graphs for debugging.

Can I use async training with colocated GPUs in slime?▼

No, colocated mode is not supported with train_async.py. Run async training with separate GPU allocations for actor training and rollout, and tune --async-buffer-size and --update-weights-interval to avoid stale weights.

How do I fix CUDA OOM during slime RL training?▼

Enable --recompute-activations for gradient checkpointing, reduce --micro-batch-size to 1, turn on --sequence-parallel, or lower --global-batch-size. In colocated mode, also reduce --sglang-mem-fraction-static to around 0.4.

Related Skills

Explore More Software Engineering Skills→
box755box755

slime-rl-training

Orchestrate Megatron-LM training with SGLang router rollouts for RL post-training.

Community
Advanced
i-bebsii-bebsi

evaluating-llms-harness

Benchmarks LLMs across 60+ academic tasks like MMLU, GSM8K, and HumanEval.

Community
Advanced
qcmuuqcmuu

openrlhf-training

Run distributed OpenRLHF RLHF training on Ray with vLLM acceleration.

Community
Advanced
davila7davila7
30.5k

torchforge-rl-training

Implements PyTorch-native reinforcement learning training workflows using torchforge, Monarch, and TorchTitan.

Community
Advanced