add-cuda-kernel

What problem does it solve?

This Skill provides a comprehensive, step-by-step guide to integrating new custom CUDA kernels into the FlashInfer library, enabling users to extend its functionality with their own optimized operations.

Core Features & Use Cases

• End-to-End Workflow: Covers defining CUDA kernels, creating C++ launchers, implementing TVM-FFI bindings, generating JIT modules, and creating Python APIs.
• Best Practices: Demonstrates proper error handling, dtype dispatch, and architecture-specific compilation.
• Use Case: A researcher needs to implement a novel attention mechanism not supported by FlashInfer. They can follow this guide to add their custom kernel, test it rigorously, and integrate it seamlessly into their existing FlashInfer-based inference pipeline.

Quick Start

Follow the detailed steps in the tutorial to add a new CUDA kernel to FlashInfer.