skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

pyvene-interventions

Perform causal interventions on PyTorch models using pyvene's declarative intervention framework.

1|Updated Mar 5, 2026
Search Tags:#pytorch#interpretability#mechanistic interpretability#activation patching#causal tracing#causal intervention#pyvene
One-click install
npx skills add https://github.com/Clay-HHK/claude-skills --skill pyvene-interventions-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: pyvene-interventions
Source: https://github.com/Clay-HHK/claude-skills/tree/main/pyvene
Command: npx skills add https://github.com/Clay-HHK/claude-skills --skill pyvene-interventions-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyvene, torch, transformers, and includes references (resource) components.

What problem does it solve? Testing causal hypotheses about neural network behavior requires complex activation patching and intervention code that is hard to reproduce and share. This Skill provides structured guidance for running causal tracing, activation patching, and trainable interchange interventions with pyvene's declarative configuration system. ## Core Features & Use Cases - Causal Tracing: Run ROME-style localization sweeps over layers and token positions to find where factual associations are stored. - Activation Patching & Circuit Analysis: Swap activations between clean and corrupted runs to identify which components drive behaviors like indirect object identification. - Trainable Interventions (DAS/IIT): Learn low-rank rotation subspaces that reveal interpretable causal directions, and share interventions via HuggingFace. - Use Case: A researcher wants to know which layers of GPT-2 encode the answer in an IOI task. They configure a VanillaIntervention per layer, patch attention outputs from a clean run into a corrupted run, and measure logit differences to localize the circuit. ## Quick Start Ask the AI to help you write a pyvene activation patching experiment that swaps layer 8 block outputs between two prompts and compares the resulting token probabilities.

Frequently Asked Questions about pyvene-interventions

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run activation patching with pyvene?▼

Activation patching in pyvene uses an IntervenableConfig with a VanillaIntervention targeting a specific layer and component. Pass clean inputs as sources and corrupted inputs as base to intervenable(), then compare output logits to measure each component's causal effect.

What is the difference between pyvene, TransformerLens, and nnsight?▼

pyvene offers declarative configs, trainable interventions, and HuggingFace sharing for any PyTorch model. TransformerLens suits exploratory transformer analysis, while nnsight supports remote execution on large models via NDIF.

Does pyvene work with models other than transformers?▼

Yes, pyvene works with any PyTorch model, not just transformers. It has been tested on GPT-2, LLaMA, Pythia, Mistral, BLIP, ESM protein models, and Mamba state space models.

How do I train a DAS intervention in pyvene?▼

Use LowRankRotatedSpaceIntervention with a low_rank_dimension in your RepresentationConfig, then optimize intervenable.get_trainable_parameters() with a standard PyTorch optimizer against a counterfactual target behavior.

Why does my pyvene intervention fail with a component name error?▼

pyvene requires exact component names like mlp_output or attention_output, not shorthand like mlp. Check the supported component list including block_input, block_output, mlp_activation, and query_output.

Can I share pyvene interventions on HuggingFace?▼

Yes, call intervenable.save_intervention("username/my-intervention") to publish, and pv.IntervenableModel.load("username/my-intervention", model=model) to reload. This makes intervention experiments reproducible across users.

Related Skills

Explore More Education & Research Skills→
box755box755

nanogpt

Trains GPT-style language models with PyTorch on Shakespeare or OpenWebText data.

Community
Intermediate
tianhao909tianhao909
1

transformer-lens-interpretability

Inspect and manipulate transformer internal activations and weights for mechanistic interpretability.

Community
Advanced
NousResearchNousResearch
239.1k

saelens

Train and analyze sparse autoencoders for mechanistic interpretability of neural networks.

Official
Advanced
Orchestra-ResearchOrchestra-Research
11.5k

nnsight-remote-interpretability

Inspect and manipulate neural network internals with nnsight and remote NDIF execution.

Official
Advanced