skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

nnsight-remote-interpretability

Interpret and manipulate PyTorch model internals locally or remotely via NDIF.

1|Updated Mar 5, 2026
Search Tags:#pytorch#remote execution#interpretability#mechanistic interpretability#nnsight#ndif#activation patching
One-click install
npx skills add https://github.com/Clay-HHK/claude-skills --skill nnsight-remote-interpretability-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: nnsight-remote-interpretability
Source: https://github.com/Clay-HHK/claude-skills/tree/main/nnsight
Command: npx skills add https://github.com/Clay-HHK/claude-skills --skill nnsight-remote-interpretability-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires nnsight, torch, and includes references (resource) components.

What problem does it solve? Running interpretability experiments on large language models requires deep access to internal activations, and models like Llama-3.1-70B or 405B exceed local GPU capacity. This Skill provides guidance for tracing, patching, and steering neural network internals with nnsight, including remote execution on massive models through NDIF. ## Core Features & Use Cases - Activation Analysis & Patching: Trace any PyTorch model, save hidden states and attention patterns, and patch activations between clean and corrupted prompts to test causal relationships. - Remote Execution via NDIF: Run identical interpretability code on 70B+ models by toggling remote=True, with no local GPU required. - Cross-Prompt Interventions & Steering: Share activations between prompts in a single trace, add steering vectors, and perform logit lens analysis. - Use Case: A researcher wants to find which layer of Llama-3.1-70B encodes factual recall. They use activation patching sweeps remotely via NDIF, comparing logit differences across all layers without owning any GPU hardware. ## Quick Start Ask the AI to help you write an nnsight script that traces a prompt through GPT-2, saves the layer 8 hidden states, and patches them into a corrupted prompt to measure the effect on predictions.

Frequently Asked Questions about nnsight-remote-interpretability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I access internal activations of a PyTorch model with nnsight?▼

Wrap the model with nnsight's LanguageModel class and use the trace context manager. Inside the trace, access module outputs like model.transformer.h[5].output[0] and call .save() to retrieve the tensor after the context exits.

How do I run interpretability experiments on 70B models without a GPU?▼

Use nnsight's remote execution through NDIF. Sign up at login.ndif.us for an API key, load a large model like Llama-3.1-70B, and pass remote=True to model.trace. The same code runs on NDIF servers and returns saved activations.

nnsight vs TransformerLens vs pyvene for interpretability?▼

nnsight supports any PyTorch architecture and remote NDIF execution with deferred tracing. TransformerLens offers a consistent API but only for transformers, while pyvene provides declarative, shareable intervention configurations. Choose based on architecture coverage and workflow needs.

Why are my saved activations empty or wrong after the trace context?▼

Values inside a trace are Proxy objects that record operations without executing them. You must call .save() on any value you need after the context exits; unsaved proxies are not accessible once tracing completes.

Does nnsight support gradient computation with remote execution or vLLM?▼

No, gradient access is not supported for vLLM backends or NDIF remote execution. For gradient-based analysis, run the model locally with standard execution using LanguageModel and device_map settings.

How do I find the correct module paths for different model architectures?▼

Module paths differ between models: GPT-2 uses model.transformer.h[i] while LLaMA uses model.model.layers[i]. Print model._model or iterate named_modules() to inspect the actual structure before writing trace code.

Related Skills

Explore More Education & Research Skills→
Clay-HHKClay-HHK
1

nnsight-remote-interpretability

Interpret and manipulate PyTorch model internals locally or remotely via NDIF.

Community
Intermediate
OpenLAIROpenLAIR
1.0k

transformer-lens-interpretability

Analyze and steer transformer activations with TransformerLens for mechanistic interpretability.

Official
Advanced
founderphantomfounderphantom

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose activations into interpretable features.

Community
Intermediate
qcmuuqcmuu

pyvene-interventions

Configure pyvene interventions for causal tracing on PyTorch models.

Community
Advanced