CS Paper Radar

From Perturbation Sensitivity to Sparse Autoencoder Feature Interpretation: An EEG Foundation-Model Case Study

Yucong Cao, Chenqi Li, Tingting Zhu

cs.LG ·submitted 2026-09-28 01:31 UTC · v2

Abstract

Sparse autoencoders (SAEs) split a model's internal activity into many sparse units, or latents, and a latent is often named after a concept when removing that concept from the input changes its activity. We identify a consequential confound in this inference: if removing the concept changes the input more than the comparison edit does, a larger response may reflect a larger edit rather than the concept. Brain-signal (electroencephalography, EEG) models provide a controlled test, because how much an edit changes the EEG input can be measured directly. We measure the SAE response as the change in how often each latent is active, averaged over all latents. Across 27 combinations of model, dataset, and layer, removing the 8--12 Hz alpha band changes this response 7.33 times as much as removing an equally wide 30--34 Hz control band, but it also changes the signal far more. Matching the two edits per trial on squared time-domain distortion lowers the mean ratio to 1.42, with 19 of 27 combinations above one. Matching thus substantially reduces, but does not eliminate, the alpha--control contrast. The residual supports relative sensitivity under this single control, not general spectral specificity or an individual alpha-representing latent. We use an evidence-to-claim map to separate what this comparison supports from stronger claims of specificity and feature semantics.

Comment
Revised version with expanded experimental analysis and discussion of limitations

Closest papers

Nearest neighbours by meaning across the whole index.

Controlled Evaluation of EEG Foundation Models: Random Initialisation, Band-Power and Validation-Selected Probes

cs.LG cs.AI ·2026-10-23 ·Mingyu Jeon, Suwan Cho, Jae Young Suh
similarity 0.64
Abstract

EEG foundation models report downstream gains, yet their protocols seldom include three controls. These are the same encoder untrained, spectral features fitted on the same data, and a layer picked without looking at test scores. We therefore evaluate five publicly released EEG foundation models and an in-house JEPA-style model under one protocol with all three controls and several seeds, on five benchmarks and four added task families. The controls change the reading in three ways. First, picking the probed layer by its test score raises the layer-wise peak by an average of +0.034 and reverses the peak-versus-output gap in 9 of 36 cells. Second, with the layer picked on validation, random initialisation matches or beats pretraining for half of the models on sleep and emotion, and no model beats a five-band log-power baseline on emotion or motor imagery. Third, after full fine-tuning only 5 of 30 pairs gain beyond twice their seed spread, and the one clear loss of -0.290 AUROC persists under matched learning rates. From these results we draw four reporting practices, and we release every cell for re-analysis.

0.06

Where You Measure Decides What You Measure: Position Selection in Ablation-Based SAE Evaluation

cs.LG ·2026-08-13 ·Valentin Noël
code/data similarity 0.64
Abstract · 19 pages, 3 figures. Code and data: https://github.com/vcnoel/sae-artifact

Sparse autoencoders are meant to name the things a language model computes, and the usual way to check that a latent matters is to switch it off and see what changes. But a latent fires at many tokens, and the effect has to be measured at one of them. The convention is to measure where the latent fires hardest. That choice is almost never reported, and it is not made by the experimenter: it is made by the dictionary under evaluation. Change the dictionary and the measurement moves to a different token. We show this is not a detail. Take two sparse autoencoders released by Google for the same model and match their latents by decoder similarity: even among the pairs the two dictionaries encode almost identically, they pick different tokens for a large share of them. Two dictionaries compared under the usual protocol are therefore very often compared at different places. To separate the convention from the dictionaries we train six autoencoders from one initialisation, differing only in fitting choices, so that a latent means the same thing in each. Most of the variance such a comparison reads as "these dictionaries disagree about this latent" turns out to be the position instead: it falls from 7.6% and 11.9% of variance to near zero once every dictionary is measured at the same token. More evaluation data does not rescue it. Across a sixteenfold range of corpus sizes the dictionaries agree less about where to measure, not more, so the problem grows with scale. The correction is one line of evaluation code. We give the protocol an ablation-based causal number must report to be comparable across papers, and an audit of five published papers against it. In short: a causal number reported without its position describes the token it was taken at as much as the latent it was taken from.

0.30

What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions

cs.LG cs.CLstat.ML ·2026-10-11 ·Yifeng Guo, Yanran Li, Xiang Chen
similarity 0.63
Abstract

Sparse autoencoder (SAE) features are typically discovered from how their activations co-vary with behaviour, yet they are often used through interventions such as steering or ablation. We take intervention attribution, the first-order effect of a specified intervention on the model output, as our estimand. For a broad class of response functions, we show that observational feature-discovery evidence generally does not identify this attribution, and derive an exact gradient-based decomposition of the gap between linear-probe predictions and intervention attribution. Across four model-SAE pairs, median probe-to-attribution magnitude ratios range from tens to hundreds, while attributions closely track small-step steering. For single-latent ablations with nonzero effects, gradient predictions match the realized sign in 91-97% of cases, compared with 17-35% for probe-implied changes. Attribution-selected sets also produce larger absolute mean ablation effects on held-out prompts than sets selected by correlation, rank tests or probe coefficients. Together, these results establish intervention attribution as a principled, gradient-estimable target for selecting SAE features for intervention.

0.17

Massive Activations Confound Sparse Autoencoders

cs.LG cs.AI ·2026-11-10 ·Nathan Delisle, Chenhao Tan
publishedcode/data similarity 0.60
Abstract · Accepted at NeurIPS 2026 (poster). Code: https://github.com/nathandelisle/massive-saes. SAEs: https://huggingface.co/nathandelisle/massive-s…

Sparse autoencoders (SAEs) are commonly evaluated by variance explained and ablation damage. We show how massive activations undermine both evaluations, and propose a correction. Massive activations cause up to 99.9% of activation variance to lie along the first principal component (PC1). Because the PC1 component has no unique sparse decomposition, each SAE run rebuilds it from a different set of features. These features dominate importance rankings: an average of 47 of the 50 most important features are unmatched in a separate run, compared with about 27 of a random 50. A dictionary can also report 99.96% of variance explained while leaving 33% of the variance outside PC1 unexplained. We bypass PC1 by training the SAE on activations with PC1 removed, then adding PC1 to the reconstruction. In affected models, the top-50 feature sharing rate rises from 4-39.5% to 73-99.5%. At the same TopK sparsity, the resulting SAE degrades the model up to 44% less and reduces content reconstruction error by up to 33.9%.

0.27

Spectral Considerations on Unsupervised Time-domain EEG Representation Learning with Deep Networks

cs.LG ·2026-10-12 ·David Aquilué-Llorens, Abel Puigseslloses, Angel Blanco, Claire Braboszcz, Aureli Soria-Frisch
position similarity 0.58
Abstract

Electroencephalography (EEG) is a non-invasive, scalable source of quantitative biomarkers for the early diagnosis of brain disorders, and deep representation learning has become a powerful paradigm for extracting information-rich latent features from raw signals. However, neural networks tend to under-represent high-frequency information, a concern that is especially acute for EEG given its oscillatory nature and characteristic $1/f^χ$ power decay, which risks masking disease-relevant high-frequency signatures. In this study, we systematically investigate the spectral constraints of unsupervised time-domain feature extraction across three autoencoder architectures (LSTMVAE, VAEEG, Transformer) and a state-of-the-art EEG foundation model. Using synthetic experiments and empirical validation, we show that while autoencoders capture macro-scale shifts such as alpha slowing, they remain biased against higher frequencies due to a magnitude-driven masking effect induced by the Mean Squared Error loss under 1/f-decaying spectra. Increasing latent dimensionality mitigates this attenuation but sacrifices latent compactness and generative stability. We argue that, as the field advances toward large-scale foundation-level feature extractors, it is imperative to move beyond simple time-domain reconstruction and to integrate spectral-domain loss terms and gradient-balancing techniques, so that the subtle high-frequency transients critical for diagnosing complex brain disorders are preserved in the latent representation.

0.09

Sparse Autoencoder Attribution Graphs Are Decided at the Margin

cs.LG cs.AIcs.CL ·2026-10-31 ·Karl Li
similarity 0.57
Abstract · 34 pages, 8 figures

Attribution graphs over the features of sparse autoencoders (SAEs) are used to predict what an ablation does and are compared across training seeds. A TopK SAE keeps the largest preactivations at each token, and a feature's margin is its preactivation's distance to the cutoff. On five public TopK SAE pairs from three Pythia models, zeroing one feature moves a next layer feature across the cutoff in 0.74 of cases or more, and the crossings carry most of the response's energy. Over writes that scale a feature's code by three factors, the graph built on the features active beforehand has a median relative error of 0.247 or more. The margin graph adds the first order change to every preactivation and reranks, at the same cost, and errs by at most 0.033. At the median, the crossing features' decoded change is 1.24 times the model's own change in norm or more. Between seeds of one testbed, sparse maps between features explain 0.85 of a graph's energy on its active features, and 0.47 once the other seed's graph keeps only its own. The largest model studied is pythia-160m.

0.28

When Are Two SAE Latents the Same Feature?

cs.LG ·2026-11-03 ·Alexandra Suvorikova, Ilgam Latypov, Anton Trukhan, Yuriy Dorn
similarity 0.57
Abstract

Comparing sparse autoencoders (SAEs) requires connecting feature correspondence, differences between whole dictionaries, and uncertainty from finite evaluation data. We develop a framework that connects these tasks through an energy-weighted optimal-transport plan under a specified similarity geometry. The same plan defines a dictionary-level discrepancy, decomposes it into latent contributions, and quantifies how each latent's mass is distributed across counterparts. We characterize relationships between decoder, activation, and first-order causal similarities; idealized models clarify the interpretation of splitting and absorption. For fixed trained SAEs, we derive joint asymptotic inference for transport costs and fractions, accounting for activation similarities and both energy marginals estimated from the same documents. A joint document bootstrap gives asymptotically valid simultaneous confidence bands. Experiments across four language models locate cross-seed discrepancy and separate context-dependent changes.

0.20

Neural State Prediction: Obstructing Shortcut Learning in EEG Foundation Models

cs.AI ·2026-09-25 ·Kieren Yu, Ziyang Liu, Chang Huang, Jintai Chen, Kaishun Wu
similarity 0.56
Abstract

EEG foundation models increasingly use masked prediction to learn from unlabeled recordings, but optimizing this objective does not ensure transferable neural representations. A central challenge is that stable positional cues and local correlations can make masked regions predictable without integrating distributed neural context. To reduce this reliance on low-information prediction paths, we introduce Neural State Prediction (NSP), a latent-predictive framework that constrains both the prediction target and the available context. NSP uses a Target Encoder updated by an exponential moving average (EMA) to define latent supervision. Identity residualization removes additive effects associated with channel identity and relative time from the targets, while topology-separated context excludes their immediate spatial and temporal neighborhood from the visible input. We pretrain NSP on 2.2 million EEG segments from TUEG and evaluate it across 30 downstream datasets spanning clinical diagnosis, sleep staging, emotion recognition, motor imagery, event-related potentials, cognitive-state decoding, and language retrieval. Under full-parameter multi-task fine-tuning on EEG-FM-Bench, NSP achieves 63.94 macro balanced accuracy across 14 datasets, exceeding the strongest evaluated baseline by 2.35 percentage points. Controlled component ablations assess the contribution of each mechanism, while matched context controls and held-out interventions characterize the role of context geometry, signal content, and positional information. Jointly designing latent targets and their context offers a promising direction for EEG foundation models that learn from distributed signal structure.

0.21