skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

mamba-architecture

Implements and benchmarks Mamba selective state-space models for linear-complexity sequence modeling.

1|Updated Mar 5, 2026
Search Tags:#inference#mamba#ssm#long context#language model#linear complexity#state space model
One-click install
npx skills add https://github.com/Clay-HHK/claude-skills --skill mamba-architecture-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: mamba-architecture
Source: https://github.com/Clay-HHK/claude-skills/tree/main/mamba
Command: npx skills add https://github.com/Clay-HHK/claude-skills --skill mamba-architecture-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires mamba-ssm, torch, transformers, causal-conv1d, and includes references (resource) components.

What problem does it solve? Transformers scale quadratically with sequence length and require large KV caches, making long-context inference slow and memory-hungry. This Skill provides working code and guidance for using Mamba state-space models, which achieve O(n) complexity, 5× faster inference, and constant memory per token. ## Core Features & Use Cases - Mamba Block & LM Construction: Build Mamba-1 and Mamba-2 blocks or full language models with MambaLMHeadModel, configuring d_state, d_conv, expand, and multi-head parameters. - Pretrained Model Inference: Load state-spaces/mamba models (130M to 2.8B) from HuggingFace and generate text with temperature, top-p, and repetition penalty controls. - Benchmarking & Training Guidance: Compare generation speed against Transformers and follow distributed training, mixed precision, and gradient checkpointing recipes from the references. - Use Case: You need to serve a language model over 100K-token genomic sequences on limited GPU memory. Use this Skill to configure a Mamba-2 model that processes the sequence linearly without a KV cache. ## Quick Start Ask the assistant to create a Mamba-2 language model with MambaLMHeadModel and generate text from a prompt on a CUDA GPU.

Frequently Asked Questions about mamba-architecture

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I use a pretrained Mamba model from HuggingFace?▼

Load pretrained Mamba models with MambaLMHeadModel.from_pretrained, passing a model name like state-spaces/mamba-2.8b, plus device and dtype. Use a compatible tokenizer such as EleutherAI/gpt-neox-20b, then call model.generate with temperature and top_p settings.

What is the difference between Mamba-1 and Mamba-2?▼

Mamba-1 uses d_state=16 with a single-head structure, while Mamba-2 uses d_state=128 with a multi-head structure, RMSNorm, and tensor parallelism support. Mamba-2 is configured via the Mamba2 class with headdim and ngroups parameters.

Does Mamba require a GPU and CUDA to run?▼

Yes, Mamba requires Linux, an NVIDIA GPU, PyTorch 1.12+, and CUDA 11.6 or higher. The mamba-ssm package relies on custom CUDA kernels, and causal-conv1d should be installed separately for efficient convolution operations.

Why does Mamba use less memory than Transformers during inference?▼

Mamba stores a fixed-size recurrent state of roughly d_model × d_state per layer instead of a growing KV cache. This gives constant memory per token regardless of context length, while Transformer KV caches grow linearly and can reach tens of gigabytes.

How do I fix CUDA out of memory errors when training Mamba?▼

Reduce the per-GPU batch size, shorten the sequence length, or enable gradient checkpointing on the model. Gradient checkpointing saves roughly 30-40% of activation memory with minimal speed impact during training.

When should I use Transformers instead of Mamba?▼

Use Transformers when you need best-in-class task performance and have sufficient compute, since Mamba trades some capability for efficiency. Mamba fits better for 100K+ token sequences, streaming applications, and memory-constrained inference.

Related Skills

Explore More Education & Research Skills→
NVIDIA-BioNeMoNVIDIA-BioNeMo
413

kermt-continue-pretrain

Automate KERMT pretraining continuation with checkpoint validation and distributed training execution.

Official
Advanced
yo-stevenyo-steven

deployment

Deploy HF LLM checkpoints as OpenAI-compatible endpoints via vLLM, SGLang, or TRT-LLM.

Community
Advanced
OpenLAIROpenLAIR
1.1k

aris-system-profile

Profiles scripts, processes, GPUs, memory, and interconnects to produce structured performance reports.

Official
Advanced
jonnabiojonnabio
6

rwkv-architecture

Process and generate text using the RWKV model with PyTorch.

Community
Advanced