mamba-architecture

Load pretrained Mamba models with MambaLMHeadModel.from_pretrained using identifiers like state-spaces/mamba-2.8b, paired with a compatible tokenizer such as EleutherAI/gpt-neox-20b. Then call model.generate with input_ids, max_length, temperature, and top_p.

Mamba-1 uses a state dimension of 16, while Mamba-2 uses d_state=128 with a multi-head structure, RMSNorm, and tensor parallelism support. Mamba-2 is configured via the Mamba2 class with headdim and ngroups parameters.

Mamba achieves roughly 5x faster inference than comparable Transformers, with the speedup growing as sequence length increases because Mamba scales linearly while attention scales quadratically. Mamba also needs no KV cache, cutting inference memory significantly.

Mamba requires Linux, an NVIDIA GPU with CUDA 11.6+, and PyTorch 1.12+. VRAM needs range from 2GB for the 130M model to 28GB for the 2.8B model in FP16.

Slow installs usually come from building from source; use pip install mamba-ssm --no-build-isolation for binary wheels. If causal-conv1d is missing, install it separately with pip install causal-conv1d>=1.4.0.

Avoid Mamba when you need best-in-class task performance and have ample compute, where Transformers still lead. Alternatives like RWKV, RetNet, or Hyena may fit better for RNN-Transformer hybrids, retention-based designs, or convolution-based approaches.