miles-rl-training

What problem does it solve?

This Skill provides a robust framework for training large-scale Mixture-of-Experts (MoE) models, addressing challenges in stability, low-precision training, and train-inference alignment crucial for enterprise-grade reinforcement learning.

Core Features & Use Cases

• Low-Precision Training: Supports FP8 and INT4 quantization-aware training for massive models.
• Performance Optimizations: Includes speculative RL for increased throughput and efficient weight synchronization.
• Train-Inference Alignment: Ensures consistency between training and inference pipelines using techniques like R3 and TIS.
• Use Case: Train a 1TB+ MoE model like DeepSeek V3 or Qwen3-MoE efficiently using FP8 precision on H100/H200 GPUs, ensuring bit-wise identical train-inference alignment for production deployment.

Quick Start

Use the miles skill to train a Qwen3-30b-a3b model with the grpo advantage estimator and a batch size of 512.