skills.rest
Orchestra-Research
Orchestra-ResearchOFFICIAL
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural network activations into interpretable features.

13.0k|930|Updated Nov 3, 2025
Search Tags:#superposition#feature discovery#sparse autoencoders#mechanistic interpretability#saelens#transformer-lens#feature steering
One-click install
npx skills add https://github.com/Orchestra-Research/AI-research-SKILLs --skill sparse-autoencoder-training-orchestra-research
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/Orchestra-Research/AI-research-SKILLs/tree/main/04-mechanistic-interpretability/saelens
Command: npx skills add https://github.com/Orchestra-Research/AI-research-SKILLs --skill sparse-autoencoder-training-orchestra-research

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens, transformer-lens, torch, and includes references (resource) components.

What problem does it solve? Neural network neurons are polysemantic, activating for many unrelated concepts due to superposition, which makes model internals hard to interpret. This Skill guides training and analyzing Sparse Autoencoders (SAEs) with SAELens to decompose dense activations into sparse, monosemantic features. ## Core Features & Use Cases - Pre-trained SAE Analysis: Load SAEs from releases like gpt2-small-res-jb, encode activations into sparse features, and identify top-activating features per token. - Custom SAE Training: Configure and train standard, gated, TopK, or JumpReLU SAEs with L1 warm-up, ghost grads, and W&B logging, monitoring L0, CE loss recovery, and dead feature ratios. - Feature Steering and Attribution: Compute per-feature logit contributions, steer generation by adding decoder directions to the residual stream, and ablate features to test causal importance. - Use Case: A researcher studying what GPT-2 has learned loads a pre-trained SAE on layer 8, finds features that consistently activate for France-related prompts, then steers generation by amplifying a specific feature direction. ## Quick Start Load a pre-trained SAE for GPT-2 small layer 8 and show me the top activating features for each token in the prompt "The capital of France is Paris".

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a Sparse Autoencoder on language model activations?▼

Configure a LanguageModelSAERunnerConfig with your model name, hook point, d_sae expansion factor, and l1_coefficient, then run SAETrainingRunner. Enable l1_warm_up_steps and use_ghost_grads to prevent dead features during training.

How to load pre-trained SAEs with SAELens?▼

Use SAE.from_pretrained with a release name like gpt2-small-res-jb and a sae_id such as blocks.8.hook_resid_pre. Pre-trained SAEs are also available on HuggingFace under the saelens tag and browsable on Neuronpedia.

What is the difference between standard, gated, and TopK SAE architectures?▼

Standard SAEs use ReLU with an L1 penalty for general-purpose training. Gated SAEs add a learned gating mechanism for better sparsity control, while TopK SAEs activate exactly K features per input for consistent sparsity.

Why does my SAE have a high dead feature ratio?▼

Dead features usually result from applying full L1 penalty too early. Set l1_warm_up_steps to 500-2000 to ramp the penalty gradually and enable use_ghost_grads to revive features that stop activating.

When should I use TransformerLens instead of SAELens?▼

Use TransformerLens directly for basic activation analysis or causal intervention experiments without feature decomposition. SAELens is specifically for discovering interpretable features, studying superposition, and feature-based steering.

How do I fix poor SAE reconstruction quality?▼

Lower the l1_coefficient to reduce the sparsity penalty or increase d_sae for more capacity. Target a CE loss score of 80-95% and explained variance above 90% as reconstruction quality benchmarks.

Related Skills

Explore More Education & Research Skills→
zjunlpzjunlp
75

Feature Dictionary Learning

Decompose dense LLM activations into sparse interpretable feature dictionaries.

Official
Advanced
math-incmath-inc
1.2k

sparse-autoencoder-training

Train and analyze Sparse Autoencoders on transformer activations with SAELens.

Official
Advanced
jonnabiojonnabio
6

pyvene-interventions

Performs customizable causal interventions on PyTorch neural network models.

Community
Advanced
zjunlpzjunlp
75

mechanism-skills

Route mechanistic interpretability questions to eleven method families for localizing internal model components.

Official
Advanced