sglang-installer

What problem does it solve?

This Skill streamlines installing SGLang from source, configuring dependencies, setting up CUDA environments, and launching a working inference server on NVIDIA GPUs (B200/H100/A100), reducing setup time and common misconfigurations.

Core Features & Use Cases

• From-source installation and dependency management: Install SGLang and its required libraries on CUDA-enabled GPUs, handling PyTorch, sgl-kernel, FlashInfer, and NVIDIA packages.
• Environment setup and validation: Configure CUDA_HOME, LD_LIBRARY_PATH, and related env vars; verify CUDA and PyTorch CUDA support.
• Server startup, testing and debugging: Launch the inference server, perform health checks, and diagnose common runtime issues; supports tensor parallelism configurations and MoE readiness with DeepEP if needed.
• Disaggregation readiness: Guidance for prefill/decode disaggregation using transfer backends like Mooncake or NIXL when deploying MoE models.

Quick Start

Clone and install SGLang from source, install NVIDIA libraries (nvidia-nccl-cu12 and nvidia-cudnn-cu12), optionally install Mooncake or NIXL for disaggregation, set up the environment, and start the server.