What problem does it solve? Transformer models suffer from quadratic attention complexity and ever-growing KV caches, making long-context inference slow and memory-hungry. This Skill provides guidance for using RWKV, an RNN-Transformer hybrid that trains in parallel like GPT but infers sequentially with O(1) memory per token. ## Core Features & Use Cases - Dual-mode inference: Run models in GPT mode (parallel) or RNN mode (sequential) with identical outputs, enabling streaming token-by-token generation. - Infinite context processing: Stream through million-token documents with a fixed-size recurrent state instead of a growing KV cache. - Fine-tuning and state management: Train with PyTorch Lightning and DeepSpeed, and save, load, blend, or expire conversation states for multi-session chatbots. - Use Case: Build a document summarization service that processes 100K+ token files on a single GPU, maintaining constant memory while a Transformer equivalent would require hundreds of gigabytes of KV cache. ## Quick Start Install the rwkv and torch packages, then load a pretrained RWKV model and generate text token by token using its RNN mode.
