add-jit-kernel

What problem does it solve?

This Skill provides a structured, step-by-step guide to integrate custom CUDA kernels into the SGLang framework, enhancing its computational capabilities for specific AI model inference tasks.

Core Features & Use Cases

• JIT Kernel Integration: Learn how to add lightweight, Just-In-Time compiled CUDA kernels.
• Abstractions: Utilize provided C++ and CUDA abstractions for safety, readability, and consistency.
• Workflow Guidance: Follow a complete process from kernel implementation to Python wrapping and testing.
• Use Case: Enhance an AI model's inference speed by adding a custom kernel for a novel activation function or a specialized tensor operation not supported by default.

Quick Start

Follow the tutorial to implement a new JIT kernel in the elementwise/scale.cuh file.