skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

rwkv-architecture

Implement RWKV models for linear-time inference with constant memory and infinite context.

1|Updated Mar 5, 2026
Search Tags:#fine-tuning#inference#linear attention#rwkv#rnn#long context#language model
One-click install
npx skills add https://github.com/Clay-HHK/claude-skills --skill rwkv-architecture-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: rwkv-architecture
Source: https://github.com/Clay-HHK/claude-skills/tree/main/rwkv
Command: npx skills add https://github.com/Clay-HHK/claude-skills --skill rwkv-architecture-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires rwkv, torch, transformers, and includes references (resource) components.

What problem does it solve? Transformer models suffer from quadratic attention complexity and ever-growing KV caches, making long-context inference memory-prohibitive. This Skill guides you through using RWKV, an RNN-Transformer hybrid that trains in parallel like GPT but infers sequentially with O(1) memory per token. ## Core Features & Use Cases - Streaming Text Generation: Generate tokens one-by-one with constant memory using RNN mode and persistent state. - Infinite Context Processing: Stream through million-token documents while the recurrent state accumulates context without a KV cache. - Fine-tuning & Training: Train RWKV models with PyTorch Lightning and DeepSpeed, parallelized like standard GPT training. - Use Case: Deploy a 7B chat model on a 16GB GPU that handles 100K+ token conversations without running out of memory, since the state stays at ~524KB regardless of context length. ## Quick Start Use the rwkv skill to load an RWKV model and generate text token-by-token with persistent state on my CUDA GPU.

Frequently Asked Questions about rwkv-architecture

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run RWKV inference in Python?▼

Install the rwkv package, load a model with RWKV(model=path, strategy='cuda fp16'), and call model.forward(tokens, state). Set RWKV_CUDA_ON=1 to enable the CUDA kernel for faster inference.

What is the difference between RWKV and Transformer models?▼

RWKV trains in parallel like a Transformer but infers sequentially like an RNN with O(1) memory per token. Transformers use O(n²) attention and a growing KV cache, while RWKV keeps a fixed-size recurrent state regardless of context length.

Does RWKV support long context or infinite context?▼

RWKV has no fixed context window because it compresses history into a constant-size recurrent state. You can stream million-token documents chunk by chunk, though recall is lossy compared to exact attention over a KV cache.

Why is my RWKV model losing context between forward calls?▼

Context is lost when the state returned by model.forward is not passed into the next call. Always chain calls as out, state = model.forward(tokens, state) so the recurrent state carries information forward.

How much VRAM does RWKV need for inference?▼

In FP16, RWKV needs roughly 1GB for 169M, 4GB for 1.5B, 16GB for 7B, and 32GB for 14B models. Memory stays constant during generation since there is no KV cache growth.

When should I use Mamba or Transformers instead of RWKV?▼

Use Transformers when you need the absolute best quality and have sufficient compute. Choose Mamba for state-space models, RetNet for retention mechanisms, or Hyena for convolution-based approaches; RWKV fits memory-constrained, long-context, streaming workloads.

Related Skills

Explore More Software Engineering Skills→
Supporter09Supporter09

sentencepiece

Train BPE or Unigram tokenizers on raw Unicode text.

Community
Intermediate
RachasumanthRachasumanth

inference-deploy

Deploy trained machine learning models for inference using vLLM, TGI, Ollama, and llama.cpp.

Community
Advanced
Mang30Mang30

training-check

Detect training quality failures in Weights & Biases runs.

Community
Advanced
NzettodessNzettodess

mamba-architecture

Implements and benchmarks Mamba selective state-space models for linear-complexity sequence modeling.

Community
Advanced