QWE AI Academy Here’s the unpopular opinion: most mechanistic interpretability content you’ve seen this year is aesthetic, not useful. Pretty attribution graphs, neuroscience metaphors, tweets about how a model “thinks in French.” What’s actually happening – and what you can do this afternoon – is much more concrete than that. Researchers are applying causality theory to LLMs: they intervene on the model’s internal wires, then measure what changes. That’s it. That’s the whole game.
If you’ve been putting off learning this because it looks like PhD territory, you can now generate a real causal circuit in your browser in about five minutes. No install, no GPU.
Key takeaway upfront
The wave of interest right now – Anthropic’s July 2026 “global workspace” paper and the BlackboxNLP MIB benchmark are both pulling in the same direction – comes from one shared move: treating LLM components as causes, not just correlates. You replace an activation, you rerun the model, you see if the output flipped. If it did, you’ve localized a cause.
The two methods everyone is arguing about are activation patching and attribution patching. Pick the right one for the size of thing you’re studying and you’re 80% of the way there.
The background, kept short
The tooling changed everything. Causal mediation analysis in NLP dates to 2020 – Vig et al. adapted Judea Pearl’s framework to language models to study gender bias (published at NeurIPS; the causal mediation approach is summarized in several subsequent papers including this 2023 arithmetic reasoning survey). That was useful but slow, one model behavior at a time. What’s different now: Anthropic’s March 2025 “Tracing the thoughts” paper linked interpretable features into full computational circuits inside Claude 3.5 Haiku, then open-sourced the pipeline two months later via a partnership with Neuronpedia and Decode Research. Suddenly you can run circuit analysis in a browser on a production-scale model.
The July 2026 J-space announcement is the newest beat – Anthropic claims a small “privileged” internal region in Claude behaves like a workspace for reportable thoughts. Wild framing, but the method underneath is the same causal intervention playbook.
Activation patching vs attribution patching
Think of activation patching as swapping one wire in a circuit board and rebooting to see what breaks. Attribution patching is estimating what would break using calculus, without rebooting each time.
| Activation patching | Attribution patching | |
|---|---|---|
| Cost per intervention | 1 full forward pass | Amortized: 2 forward + 1 backward pass total |
| Accuracy | Exact (it’s a real intervention) | Linear approximation via gradients |
| Best for | Small models, careful analysis | Large models, sweeping every head/neuron |
| Fails when | Rarely – it’s the ground truth | Patching large activations (e.g. residual stream) |
Two forward passes, one backward – that’s the entire cost of attribution patching (AtP) across thousands of components, per the 2025 RelP paper. That’s why it’s the default when you’re screening every head and neuron in a large model. The catch: turns out the linear approximation holds well for small activations like individual attention head outputs, but breaks badly on the residual stream. Neel Nanda’s original write-up on attribution patching is direct about this – it’s “bad when patching in large activations like a residual stream.”
For a beginner, activation patching wins. You’ll be working with tiny models (GPT-2 Small, Gemma-2 2B) where speed doesn’t matter. You want the ground truth, not an approximation. Learn the exact method first, then reach for the fast one when you actually need it.
Walkthrough: trace a real causal circuit in your browser
The fastest way to touch this is through Neuronpedia’s hosted circuit-tracer. Zero install. Runs on Anthropic’s open-sourced pipeline. Over 7,000 attribution graphs had been generated there as of August 2025 (per the Neuronpedia landscape post).
Step 1 – Pick a prompt with a clear right answer
Causal analysis needs a target logit to measure. A prompt like “Symptom: fever, stiff neck, photophobia. The most likely diagnosis is” works well – the correct next token is something like “meningitis,” and the interesting question is how the model chains from three separate symptoms to a unified clinical label without any intermediate reasoning visible in the output. A clear expected answer means you can measure whether your intervention actually flipped the prediction.
Step 2 – Generate the graph
Head to neuronpedia.org, hit + New Graph, pick Gemma-2 2B, paste the prompt, click Generate. A few seconds later you get an interactive attribution graph – nodes are transcoder features, edges are causal influence weights.
Step 3 – Find the intermediate features
Pin the output token. Look at the top ancestors. You should find a cluster of features that activate on symptom-combination patterns – features that effectively represent “this symptom triad” as a unified concept, even though no diagnosis was ever written in the prompt. That’s the intermediate reasoning step, materialized as a group of nodes on your screen.
Step 4 – Verify by intervention
This is the causal part. Pin at least one of those intermediate nodes and hit Steer. Down-weight the feature. Rerun. If “meningitis” drops in probability and something unrelated rises, you’ve just causally verified that this feature is doing the symptom-grouping work, not just correlated with it.
Pro tip: when you find a feature that looks meaningful, always intervene both ways – down-weight it and confirm the answer breaks, then up-weight a competing feature and confirm you can steer the answer elsewhere. One-sided evidence isn’t causal evidence.
Four gotchas no tutorial mentions
- Attribution patching lies about the residual stream. If you scale up and start using AtP to sweep everything, the linear approximation breaks on high-magnitude activations. Patch attention head outputs – fine. Patch the residual stream – you’ll get plausible-looking numbers that are wrong. Verify anything critical with real activation patching.
- LayerNorm silently corrupts gradients. Gradient-based patching interacts badly with LayerNorm’s normalization factors. The workaround is to freeze them – in TransformerLens you patch
hook_scalewith cached values so no gradients flow through. Almost no beginner guide mentions this (Nanda’s original post does). - The “AI microscope” still needs a human at the eyepiece. Anthropic states plainly in their March 2025 paper that it “currently takes a few hours of human effort to understand the circuits we see, even on prompts with only tens of words.” Circuit tracing is not automated interpretability. It’s a tool that lowers the cost of a manual investigation.
- Interventions on Neuronpedia are Gemma-2-only (as of mid-2025). Per the circuit-tracer repo, the browser Steer button only works for Gemma-2 (2B) right now. You can generate graphs for Llama-3.2-1b and others, but to causally verify features on those models you’ll need to drop into the Python library.
Where the causal frame actually pays off
The best real-world example so far is hallucinations. Anthropic’s team found that Claude’s default is to refuse unfamiliar questions – but when “known entity” circuits fire, the refusal gets overridden. When they made up a person named Michael Batkin and injected familiarity signals, the model confidently invented a bio. That’s a causal chain you can now see and, in principle, block.
Correlational interpretability tells you neurons light up when the model hallucinates. Causal interpretability lets you turn off the wire that makes it hallucinate. Different game.
FAQ
Do I need to know Judea Pearl’s causality math to do this?
No. Read the docs, not the textbook. The interventions in mech interp are simpler than the do-calculus in Pearl’s books – you’re swapping activations and measuring output changes. That’s it.
Is this different from SAEs and “features”?
SAEs give you a dictionary of interpretable features. Patching tells you which of those features actually cause a specific output. They’re complementary: modern attribution graphs use transcoder features as the nodes and causal interventions as the edges. If you only know about SAEs, you know half the picture – the vocabulary without the grammar.
Can I use these tools on GPT-4 or Claude directly?
Not really. Closed-weight commercial models don’t expose activations. That’s why the entire open-source ecosystem – TransformerLens, circuit-tracer, Neuronpedia – runs on open-weight models like GPT-2, Gemma-2, and Llama-3.2. Anthropic uses these methods internally on Claude, but you can’t replicate that externally. You study smaller open models and assume the mechanisms generalize – a big assumption that the January 2025 “Open problems in mechanistic interpretability” paper flags as an active open question.
Your next move
Open Neuronpedia, generate an attribution graph for one prompt where you already know the answer, and try to break the answer by steering a single feature. If you can flip the output by editing one node, you’ve done real causal interpretability. That’s the entry ticket.