skills.rest
gagan114662
gagan114662COMMUNITY
Explore all skills by this author →
GitHub

miles-rl-training

Train large MoE models with low-precision FP8/INT4 reinforcement learning.

Updated Jan 30, 2026
Search Tags:#enterprise#reinforcement learning#fp8#sglang#megatron-lm#moe#int4
One-click install
npx skills add https://github.com/gagan114662/content_books --skill miles-rl-training-gagan114662
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: miles-rl-training
Source: https://github.com/gagan114662/content_books/tree/main/AI-research-SKILLs/06-post-training/miles
Command: npx skills add https://github.com/gagan114662/content_books --skill miles-rl-training-gagan114662

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes references (resource) components.

What problem does it solve?

This Skill provides a robust framework for enterprise-grade Reinforcement Learning (RL) training of large-scale Mixture-of-Experts (MoE) models, addressing challenges in low-precision training and train-inference alignment.

Core Features & Use Cases

  • Low-Precision Training: Supports FP8 and INT4 quantization-aware training for massive models.
  • Train-Inference Alignment: Ensures bit-wise identical alignment between training and inference.
  • Speculative RL: Achieves maximum throughput via speculative RL for faster rollouts.
  • Use Case: Train a 1TB+ MoE model like DeepSeek V3 or Qwen3-MoE using FP8 precision with guaranteed train-inference alignment and significantly improved training throughput.

Quick Start

Use the miles skill to train the qwen3-30b-a3b model with the grpo advantage estimator and the specified HuggingFace checkpoint.

Frequently Asked Questions about miles-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train large MoE models like DeepSeek V3 using FP8 precision?▼

You can train large MoE models like DeepSeek V3 using FP8 precision by applying low-precision quantization-aware training frameworks that ensure bit-wise train-inference alignment and high throughput.

What is train-inference alignment and why does it matter for reinforcement learning?▼

Train-inference alignment ensures bit-wise identical consistency between training and inference environments. It prevents performance degradation during RL rollouts by eliminating numerical discrepancies in large MoE models.

Do I need H100 or H200 GPUs for FP8 reinforcement learning training?▼

Yes, FP8 reinforcement learning training for massive models exceeding 1TB requires specific hardware like H100 or H200 GPUs to handle the computational demands of low-precision operations and speculative RL.

Can I use sglang and Megatron-LM for enterprise RL training?▼

Yes, enterprise RL training integrates with sglang and Megatron-LM alongside Ray and PyTorch to orchestrate large-scale MoE model training, utilizing sglang-router for maximum rollout throughput.

What is the best way to achieve maximum throughput for MoE model rollouts?▼

The best way to achieve maximum throughput for MoE model rollouts is by using speculative RL combined with FP8 or INT4 low-precision training to accelerate generation and training cycles.

Does INT4 quantization-aware training work for 1TB scale models?▼

INT4 quantization-aware training is supported for 1TB scale models like Qwen3-MoE. It works alongside FP8 to reduce memory footprint while maintaining training stability through train-inference alignment.

Related Skills

Explore More Software Engineering Skills→
NVIDIA-NeMoNVIDIA-NeMo
852

perf-moe-optimization-workflow

Optimize MoE model training throughput by addressing memory, communication, and compute bottlenecks.

Official
Advanced
hhhi21ghhhi21g

miles-rl-training

Optimize reinforcement learning training for large MoE models with low-precision and speculative RL.

Community
Advanced
ovachieverovachiever
52

training-llms-megatron

Train large language models with Megatron-Core using configurable 3D and 4D parallelism.

Community
Advanced
MesferAliMesferAli

miles-rl-training

Trains large MoE models with FP8/INT4 quantization and R3 alignment.

Community
Advanced