mamba-architecture

What problem does it solve?

Mamba introduces state-space models with selective mechanisms to achieve O(n) complexity, enabling long-context sequence modeling with far faster inference than Transformers and without KV caches. It targets hardware-efficient, scalable AI runtimes for million-token contexts.

Core Features & Use Cases

• O(n) inference: Linear scaling with sequence length for long sequences.
• Hardware-aware design: CUDA kernels and selective state updates optimize throughput.
• Models & workflows: Mamba-1 (d_state=16) and Mamba-2 (d_state=128, multi-head) enable efficient LLM-style blocks without attention.
• Use Case: Deploy LMs with extremely long contexts (hundreds of thousands of tokens) with streaming outputs.

Quick Start

Install prerequisites, then instantiate a Mamba block with d_model, d_state, and d_conv, and perform a forward pass on CUDA.