skills.rest
icyzh
icyzhCOMMUNITY
Explore all skills by this author →
Website𝕏@icyzhbeGitHub

slime-rl-training

Integrate Megatron-LM actor training with SGLang rollout generation for RL post-training.

1|Updated Jul 31, 2026
Search Tags:#llm#reinforcement learning#sglang#grpo#post-training#megatron-lm
One-click install
npx skills add https://github.com/icyzh/hermes-web --skill slime-rl-training-icyzh
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/icyzh/hermes-web/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/icyzh/hermes-web --skill slime-rl-training-icyzh

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve?

This skill addresses the complexity of aligning large language models using reinforcement learning by providing a unified framework that bridges Megatron-LM training with high-throughput SGLang rollout generation.

Core Features & Use Cases

  • Hybrid RL Training: Supports advanced algorithms like GRPO, PPO, and GSPO with full parallelism (TP, PP, DP, SP).
  • High-Throughput Rollout: Leverages SGLang for efficient response generation and multi-turn interaction.
  • Use Case: Use this framework to perform post-training on models like GLM-4, Qwen3, or DeepSeek V3 to improve reasoning capabilities and tool-use accuracy.

Quick Start

Execute the training process by running the train.py script with your model configuration sourced from the scripts models directory.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I perform reinforcement learning post-training on LLMs using Megatron-LM and SGLang?▼

Reinforcement learning post-training is performed by integrating Megatron-LM for distributed actor training with SGLang for high-throughput rollout generation. This framework bridges both systems to execute complex alignment tasks efficiently.

Can I use GRPO for reasoning model optimization with this framework?▼

Yes, GRPO is supported for reasoning model optimization. The framework facilitates group-relative policy optimization alongside PPO and GSPO algorithms to improve reasoning capabilities and tool-use accuracy.

Does this framework support multi-turn agentic training and asynchronous rollout buffering?▼

Yes, multi-turn agentic training and asynchronous rollout buffering are fully supported. SGLang enables efficient multi-turn interaction response generation while handling research-grade distributed training requirements.

What parallelism strategies are available for distributed RL training of large language models?▼

Distributed RL training supports full parallelism including tensor, pipeline, data, and sequence parallelism. Megatron-LM handles actor training across these dimensions to scale post-training for large models like GLM-4 and DeepSeek V3.

Do I need Ray and PyTorch to run the LLM post-training script?▼

Yes, Ray and PyTorch are required dependencies. You execute the training process by running the train.py script with your model configuration sourced from the scripts models directory to start post-training.

Related Skills

Explore More Software Engineering Skills→
aayushsoamaayushsoam
1

slime-rl-training

Automate LLM post-training with slime and Megatron-LM integration.

Community
Advanced
vasilyu1983vasilyu1983
73

ai-llm-inference

Optimize LLM inference performance and cost with vLLM and quantization.

Community
Advanced
ovachieverovachiever
52

training-llms-megatron

Train large language models with Megatron-Core using configurable 3D and 4D parallelism.

Community
Advanced
Clay-HHKClay-HHK
1

slime-rl-training

Configures RL post-training of LLMs using Megatron-LM training with SGLang rollout generation.

Community
Advanced