skills.rest
azaanaliraza
azaanalirazaCOMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Guides LLM post-training with reinforcement learning using Megatron-LM and SGLang.

Updated Apr 18, 2026
Search Tags:#reinforcement learning#distributed training#glm#sglang#grpo#megatron-lm#llm post-training
One-click install
npx skills add https://github.com/azaanaliraza/operarius --skill slime-rl-training-azaanaliraza
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/azaanaliraza/operarius/tree/main/src-tauri/bin/hermes/optional-skills/mlops/slime
Command: npx skills add https://github.com/azaanaliraza/operarius --skill slime-rl-training-azaanaliraza

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve? Setting up reinforcement learning post-training for large language models requires coordinating distributed training (Megatron-LM) with high-throughput inference (SGLang), managing data buffers, and debugging weight synchronization—tasks that are error-prone without structured guidance. ## Core Features & Use Cases - GRPO and Async Training Workflows: Step-by-step launch commands for standard GRPO, asynchronous training, and multi-turn agentic training with tool calling. - Configuration Reference: Complete argument taxonomy covering Megatron, SGLang, and slime-specific parameters with the batch-size constraint formula. - Troubleshooting Playbook: Solutions for SGLang crashes, weight sync timeouts, OOM errors, reward collapse, and checkpoint loading failures. - Use Case: Train a Qwen3-4B reasoning model with GRPO by sourcing a pre-configured model script, preparing JSONL prompt data, and launching train.py with validated parameters. ## Quick Start Ask the assistant to help you launch a GRPO training run for a Qwen3 model using slime with your JSONL prompt dataset.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train an LLM with GRPO using slime?▼

Prepare a JSONL dataset with prompt and label fields, source a pre-configured model script from scripts/models/, then launch train.py with --advantage-estimator grpo, rollout batch size, and samples per prompt. Ensure rollout_batch_size times n_samples_per_prompt equals global_batch_size times num_steps_per_rollout.

slime vs verl vs miles for RL training?▼

Choose slime for Megatron-LM native training with SGLang inference and GLM model support. Use miles for enterprise-grade stability features, verl for flexible backend swapping, or torchforge for PyTorch-native abstractions.

Which models does slime support for RL post-training?▼

slime supports GLM-4.5/4.6/4.7, Qwen3 (4B, 8B, 30B-A3B), DeepSeek V3/R1, Llama 3 (8B, 70B), Kimi K2, and Moonlight-16B. Each model has pre-configured scripts in the scripts/models/ directory.

Why does SGLang crash during slime training?▼

SGLang engine crashes mid-training usually stem from memory pressure or batch overload. Enable --use-fault-tolerance, raise --sglang-mem-fraction-static to 0.85, or reduce --rollout-batch-size to 16.

Can I use colocated mode with async training in slime?▼

No, colocated mode is not supported with train_async.py. Async training requires separate GPUs for training and rollout; remove the --colocate flag and allocate distinct GPU counts for actor and rollout.

How do I fix CUDA OOM during slime training?▼

Enable gradient checkpointing with --recompute-activations, reduce --micro-batch-size to 1, enable --sequence-parallel, or lower --global-batch-size. In colocated mode, also reduce --sglang-mem-fraction-static to 0.4.

Related Skills

Explore More Software Engineering Skills→
thinking-machines-labthinking-machines-lab
4.0k

datasets

Construct and normalize ML datasets across JSONL, HuggingFace, and conversation formats.

Official
Intermediate
i-bebsii-bebsi

evaluating-llms-harness

Benchmarks LLMs across 60+ academic tasks like MMLU, GSM8K, and HumanEval.

Community
Advanced
castform-aicastform-ai
49

verify-environment

Validate Castform training environments by interpreting scorecards and reward variance.

Official
Advanced
tadod12tadod12

verl-rl-training

Configure scalable Verl RL training workflows for LLMs across vLLM and Megatron backends.

Community
Advanced