CS Paper Radar

Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit

Sai Adith Senthil Kumar

cs.LG ·submitted 2026-08-27 15:38 UTC · v2

Abstract

One approach in mechanistic interpretability explains model behavior through circuits: the components and connections that carry it. Frozen discovery often returns hundreds of edges, making them hard to inspect, compare, or verify exhaustively. We introduce Circuit Condensation, which post-trains models to concentrate behaviors into smaller causal graphs. Each round prunes low-attribution edges and trains a low-rank adapter to match the original through what remains. It keeps the cut only if task accuracy and held-out perplexity remain within tolerance. Across four behaviors and eight models, condensed circuits are smaller than the strongest published frozen-weight baseline in 30 of 32 settings, by 8.1x on average and up to 316x. Repeating the search without weight updates produces larger circuits in 29 of 32 settings, showing that weight updates drive the reduction rather than search alone. For 19 small circuits, exhaustive subset tests show that 11 are exactly minimal and identify removable edges in the rest. Pair ablations reveal dependencies that single-edge tests miss. On indirect object identification, condensation isolates 24 heads, including 17 with documented roles, whereas a matched frozen circuit needs 61 heads, 36 without documented roles. The result is a sufficient subcircuit of the published mechanism rather than a reconstruction of it. The resulting circuit tracks the original model's next-token distribution and, at matched edge count, predicts its errors better than a frozen circuit in 10 of 11 evaluable settings.

Comment
31 pages, 5 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability

cs.LG ·2026-10-01 ·Chuqin Geng, Li Zhang, Haolin Ye, Mark Zhang, Luke Zhang +1 more
benchmark similarity 0.69
Abstract · 35 pages, 3 figures

Mechanistic interpretability aims to recover the internal computations responsible for model behavior. Progress in automated circuit discovery is often framed as a search problem: better attribution or optimization should identify better mechanisms. This assumes that the evaluation objective can recognize a better circuit once it is found. Yet standard intervention-based evaluation replaces signals outside a circuit to isolate its effect, changing the computational environment in which the retained components operate. We call this change context distortion. This raises the question of whether faithfulness rewards recovery of model behavior or compatibility with the altered context. We show that intervention-based faithfulness can prefer an equally sized circuit that reproduces the model's behavior less well. We call this reversal misranking, an objective-level recovery gap. To isolate evaluation from discovery, we construct equally sized candidates through controlled edits to human-identified circuits in pretrained language models, including IOI and Greater-Than, and benchmark circuits with known mechanisms. We find misranking even without a discovery algorithm. Misranking also occurs in real automated circuit-discovery settings. Among outputs of EAP, EAP-IG, ACDC, and Edge-SP, KL faithfulness under ordinary resampling prefers the behaviorally worse candidate in 13.2%-24.4% of candidate pairs across these methods on four pretrained-language-model tasks. Finally, restoring selected intact-model signals reverses 96 of 100 selected persistent misrankings, demonstrating that context distortion contributes to these failures. In light of these findings, we call for rethinking both how circuit discovery is performed and how its results are evaluated.

0.17

Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMs

cs.LG cs.CL ·2026-09-22 ·Shubham Santosh Pandere, Gautam Ranka, Ritika Varshney, Navya Deshmukh, Roushni Sareen +1 more
similarity 0.63
Abstract · Accepted at BlackboxNLP 2026, Co-located with EMNLP 2026

In language models, the choice between believing the prompt and believing the weights is made by a handful of identifiable attention heads. Instruction tuning changes how models behave under conflict, but whether it rewires the underlying circuit or merely gates/reweights already present components, remains unknown. We provide the first mechanistic base-vs-instruct comparison of conflict-resolution circuits, across three families (Llama-3.2-3B, Qwen-2.5-3B, Gemma-3-4B). Five independent methods, node and edge attribution, superposition role analysis, causal ablation, and path patching, converge on gating, with the same heads, in the same late-layers, are found to be reweighted rather than replaced with a high node overlap (0.60-0.82). Behaviorally, tuning shifts models toward parametric memory, making instruct models reject a terse counterfactual context far more than base ones, the opposite of a naive user-following expectation. Yet this added skepticism is a factor of framing since it disappears when the same false claim is delivered as a coherent, evidential passage. The robustness that instruction tuning buys against terse injection is therefore real but narrow. More broadly, we believe that because the conflict circuit is preserved rather than rebuilt, interpretability and control tools calibrated on base models should transfer directly to their deployed instruct siblings.

0.34

Representational Simplicity and Circuit Size Dissociate in a Threshold-Dependent Way: A Controlled Test via Adversarial Training

cs.AI ·2026-09-27 ·Adam Elimadi
similarity 0.63
Abstract · Under review at ICLR 2027. 21 pages, 5 figures, 12 tables

Sparse-autoencoder decomposability and concentrated feature attribution are increasingly treated as evidence that a model's computation is easier to reverse-engineer. Whether this representational and attributional cleanliness actually predicts a smaller or more tractable causal circuit remains an open question. We test this directly using adversarial training as a controlled instrument: it reliably reshapes internal representations, but this alone does not constitute a test of circuit size. We investigate this question through reverse-engineering complexity: the causal structure required to recover a model's behavior at a fixed level of faithfulness. To our knowledge, this is the first controlled empirical test of whether representational or attributional simplicity translates into causal simplicity at the circuit level. Starting from the same pretrained GPT-2 Small checkpoint, we apply matched standard and adversarial continual training, requiring both conditions to retain competence on indirect object identification and pass independent robustness verification before comparing mechanisms. We then compare the models along three complementary axes: sparse-autoencoder decomposability, SAE feature engagement in task attribution, and the size of faithful circuits recovered from the raw computational graph. The robust model is more SAE-decomposable and engages fewer SAE features in task attribution. Circuit size is regime-dependent: on competence-matched IOI, standard leads or ties below 85% faithfulness, but robust needs substantially fewer edges at high faithfulness (90%, 95%), a pattern established on the primary pair while representational trends generalize across a seven-point sweep and a second corpus.

0.36

Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?

cs.LG cs.AI ·2026-09-28 ·Li Zhang, Chuqin Geng, Mark Zhang, Chen Yang, Luke Zhang +2 more
benchmark similarity 0.62
Abstract

Mechanistic interpretability (MI) aims to explain a model's behaviour through analyzing its internal computations; circuit-based explanations aim to isolate these computations with compact subnetworks validated by ablating the rest of the model. We show that circuits validated this way may fail to recover the underlying mechanism of the model's behaviour by closely reproducing its successful decisions while failing to account for most of its errors. Such explanations should account for the model's particular errors as well as its successes. We evaluate this requirement by measuring exact answer agreement separately on model successes and failures, across circuit sizes and ablation settings, on IOI, Docstring, and six model-task settings from the Mechanistic Interpretability Benchmark. We discover that many tested circuits closely replicate correct behaviour while missing most of the model's errors. On indirect object identification (IOI) for GPT-2 small, under mean ablation, the manual circuit and tested automated circuits, including one trained against the model's full output distribution, agree with the model on 97.3-99.5% of prompts it answers correctly but only 11.4-41.7% of errors. An IOI case study shows that lost errors are recoverable by restoring omitted attention-heads which raise error reproduction from 14.2% to 75.1% on a separate held-out set with 0.41 percentage point decrease on correct agreement, exceeding matched random extensions and scalar-biased control. Intervention traces show how omitted computations produce specific wrong answers for a reproducible subset of errors. In all, these findings show circuits can preserve task success without adequately explaining model's failures, and support exact error reproduction as a necessary, but not sufficient, test of circuit-based explanations of model behaviour.

0.22

How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution

cs.CL cs.LG ·2026-08-27 ·Vedant Palit, Florent Draye, Terry Jingchen Zhang, Bernhard Schölkopf, Zhijing Jin
top venuecode/data similarity 0.61
Abstract · Accepted to EMNLP 2026 (Main Conference). 29 pages, 18 figures, 9 tables

Transcoder attribution graphs are usually trained to explain why a model assigns high probability to a particular next token. We introduce Concept-Targeted Attribution (CTA), which instead trains attribution graphs with respect to a linear probe direction. CTA therefore yields probe-specific circuits that explain why an internal concept representation arises in a prompt, independently of whether it is expressed in the generated token. Using Cross-Layer Transcoders, we show that these probe-targeted graphs contain predictive structure: graph-level features predict probe accuracy across four widely studied concept categories ($ρ= 0.91$, $R^2 = 0.84$), while local features identify the sparse components driving per-prompt classification. This connects probe performance to interpretable circuit structure, allowing us to ask not only whether a probe works, but which internal computations make it work. Causal ablations further show that probe-targeted and logit-targeted graphs capture functionally distinct mechanisms. Removing probe-relevant features reduces internal concept scores while largely preserving generated tokens, whereas removing logit-relevant features changes the generated token in 92% to 100% of cases with near-zero effect on probe scores. CTA provides a framework for moving from behavioral probe accuracy to mechanistic explanations of probe performance, enabling more detailed audits of internal concept representations, including safety-critical ones. Our code is available at https://github.com/vedantpalit/concept-targeted-attribution

0.62

Attention as a Routing Graph: Live Circuit Extraction from a Single Forward Pass

cs.AI cs.LG ·2026-09-21 ·Ash Manvi, Samreena Tajreen
code/data similarity 0.61
Abstract · 7 Pages, 5 Figures, 1 table, Code: https://github.com/Aquinf03/live-circuit-routing

Finding circuits in language models usually means running many careful interventions. We try something simpler: treat attention as a routing map from one forward pass, keep a small set of routes that point toward the answer, and ask whether those routes actually matter. They often do. On induction and IOI (tasks where the "right" circuit is already known), ablating our extracted edges hurts the model much more than ablating a random set of the same size. We evaluate n=100 prompts per cell on GPT-2 Small, GPT-2 Medium, and Pythia-410M, with paired gap tests and bootstrap confidence intervals. The extract step costs one forward; a head-by-head patch sweep costs about two orders of magnitude more. We are not claiming a complete circuit atlas. We are claiming a cheap sketch that carries real causal signal on known tasks, with clear failure modes when it does not. Code and evaluation artifacts are at https://github.com/Aquinf03/live-circuit-routing.

0.45

What Should Data Teach? Moving Bottlenecks Across Circuit, Store, and Use

cs.LG ·2026-09-26 ·Yixiao Chen, Ke Cheng, Yue Liu, Jun Zhang, Jie Jiang
hot direction similarity 0.59
Abstract

What should data teach a language model at a particular point in training? A circuit view reveals three distinct bottlenecks: forming a computation, making its required content available, and selecting among available routes. A shared diagnosis-to-data principle connects them: localize the missing operation, preserve its causal relation, vary shortcut-bearing context, and re-audit the residual. Formation-sensitive selection and prerequisite ordering accelerate a binding-matching-transport path; a brief early prefix from the same training multiset retains a validation advantage through 100B tokens. Availability counterfactuals then distinguish writing content from invoking available memory, while paired supervision and context-opportunity ranking improve matched route decisions and long-context answer likelihood. A continuous 350M-model experiment connects the three interventions on the same facts: early circuit training improves subsequent learning, and the complete sequence outperforms stage-replacement controls on facts withheld from Use teaching. Independent query surfaces and opposed-source decisions expose conditional arbitration as the remaining frontier. Together, these results show why a change in the limiting operation calls for a change in supervision, not merely a new ranking of difficult examples.

0.31

Slaying the Hydra: Interaction-Aware Circuit Discovery in Language Models

cs.LG cs.AIcs.CLstat.ML ·2026-10-02 ·Sankaran Vaidyanathan, Rafal Urbaniak, Emily Bunnapradist, Michelangelo Naim, Daniel Waxman
similarity 0.59
Abstract

Localizing behavior to individual components of a language model is a central goal of mechanistic interpretability. However, scoring components one at a time misses context-dependent effects: a primary component can inhibit the activation of a backup, leading to issues with ranking components. Actual causality studies the structure of such interactions via witnesses: variables that provide contextual information to resolve interaction terms. However, estimation with witnesses typically requires combinatorial enumeration and is infeasible in practice. We introduce the witness-integrated set effect (WISE), a family of causal estimands that build on the witness mechanism while taking expectations over sets of causes and witnesses to remain computationally feasible. Building on this approach, we introduce JuntaLearner, a gradient-based circuit discovery method that learns to rank components by their causal impact across varying-sized sets of components and witnesses. Alongside faithfulness metrics, we introduce measures of necessity and task specificity, and the circuit recognition score (CRS) to summarize each metric across circuit sizes while emphasizing effects achieved by small circuits. Across tasks and models of increasing size, JuntaLearner achieves higher mean CRS compared to attribution baselines on all metrics. Since its cost does not grow with the number of candidate components, JuntaLearner scales to large models while accounting for set-level interactions and avoiding first-order approximations.

0.20