CS Paper Radar

Sparse Prototype Code Underlies Classification and Prediction Across Modalities

Yehonatan Avidan, Daniel D. Lee, Haim Sompolinsky

cs.LG cond-mat.dis-nncs.AIstat.ML ·submitted 2026-08-16 08:53 UTC · v1

Abstract

Neural representations have become a central tool for studying the internal mechanisms of modern AI models, yet their complex high-dimensional structure makes them difficult to interpret. We show that classification tasks give rise to a universal representational geometry, shared across state-of-the-art models in vision, audio, and language processing. The key structure is that within-class variability is not random in representation space. Instead, its classifier-relevant component has strong and structured correlations with the class's own centroid and with the centroids of its competing classes. Building on this observation, we derive an analytical mean-field theory governed mainly by the variability along true-class and rival-class centroid coordinates, together with a global renormalization of the class radius that compensates for the non-Gaussian statistics of real representations. The theory accurately predicts classification accuracy across architectures and modalities. The relevant geometric quantities improve systematically with model scale, mirroring the observed gains in accuracy. A striking feature of the theory is its sparsity: accurate prediction requires only a small set of centroid coordinates associated with the true class and its strongest rivals - connecting our framework to sparse-feature extraction approaches such as sparse autoencoders. Together, these results provide a parsimonious predictive theory of neural representations and suggest that classification in deep networks is governed by a sparse, centroid-aligned structure embedded within the full high-dimensional representation space.

Comment
33 pages, 13 figures, 14 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Stable Dimensionality Reduction via Isometric Centroid-Encoder

cs.LG cs.CV ·2026-10-24 ·Tomojit Ghosh, Siva Rama Krishna Prasad Changala, Prathyusha Kanakamalla, Harigovind Harikumar
similarity 0.63
Abstract · Total 45 pages, 15 figures and 16 tables

Many supervised dimensionality reduction methods promote intra-class compactness and inter-class separation but do not explicitly constrain class geometry, leading to variability in the relative arrangement of classes across independent runs. Such variability limits the reproducibility and interpretability of low-dimensional embeddings. We propose the Isometric Centroid-Encoder (ICE), a supervised dimensionality reduction framework in which class centroids are learned directly in the latent space while preserving a prescribed pairwise class-centroid geometry. Unlike existing centroid- or prototype-based methods that primarily encourage intra-class compactness, ICE explicitly regularizes the relative arrangement of latent class centroids. Unlike multidimensional scaling and many metric-learning approaches that preserve sample-level relationships, ICE operates on class centroids, preserving inter-class geometry without requiring large numbers of sample-pair or triplet constraints. We further show that the intrinsic dimensionality required to represent the centroid-distance configuration of $M$ classes is at most $M-1$, explaining the observed saturation of distance-preservation measures as the latent dimensionality increases. To quantify embedding reproducibility, we introduce the Centroid Neighborhood Stability (CNS) metric, which measures the consistency of class-level neighborhood topology across independent runs and complements existing embedding quality measures. Experiments on image and biological datasets demonstrate that ICE produces stable embeddings with improved preservation of class-centroid geometry while remaining computationally efficient. Additional analyses of optimization dynamics, sensitivity, and preprocessing further provide practical insights into latent dimensionality and explicit geometry modeling for supervised dimensionality reduction.

0.31

Global Divergence, Local Convergence: Representation Geometry in SSMs and Transformers

cs.CL ·2026-09-08 ·Amit Ben-Artzy, Roy Schwartz
top venue similarity 0.62
Abstract

Recent state-space models (SSMs) such as Mamba achieve language modeling performance comparable to transformers despite relying on fundamentally different architectures. This raises an important question: how do these structural differences influence the geometry and functional nature of their internal representations? We study this question through a multi-scale analysis of representations in transformers, SSMs, and hybrid architecture. First, we find that SSMs distribute their representational information evenly across all dimensions, whereas transformer representations are heavily dominated by a single principal direction. By evaluating hybrid architectures, we observe that the representation space becomes increasingly skewed toward a single dominant direction after each attention layer. Next, we explore how the different geometric spread of representations impacts representational capacity through compressibility. Surprisingly, we find that despite their contrasting geometric structures, both architectures exhibit tightly matched effective capacities. We further investigate whether this skewed geometry affects how concepts are encoded. Using rank-constrained probes, we demonstrate that both architectures encode concepts in subspaces of surprisingly similar dimensionality. Furthermore, we demonstrate that the transformers' dominant principal direction does not inherently encode more conceptual information. Finally, we zoom in and examine the alignment between manifolds, either by analyzing representations of specific topics or by looking at the nearest neighborhoods of tokens, and find that they are highly aligned. Ultimately, our analysis suggests that while transformers and SSMs induce different usage of latent space, they display a striking functional convergence at the level of local semantic manifolds.

0.44

Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders

cs.LG ·2026-09-21 ·Rithin Nagaraj, Rupa Laalasa Oruganti, Prerna Subhashchandra Kunder, Ashwini M Joshi
similarity 0.61
Abstract · Contains about 6 pages

The quadratic scaling of Transformer self-attention has driven the adoption of sub-quadratic Selective State Space Models (SSMs) like Mamba, which compress past context into a fixed-size recurrent hidden state. This strict informational bottleneck raises a foundational question for mechanistic interpretability: do SSMs and Transformers learn fundamentally distinct latent representations? In this work, we employ Sparse Autoencoders (SAEs) to conduct a large-scale, feature-level correspondence analysis between Mamba-130m and Pythia-70m over a 10-million token corpus. Contrary to hypotheses predicting widespread architectural divergence, we find no evidence of systematic representational divergence between architectures: across the observed Jaccard distribution, 99.98% of Mamba features cluster toward the upper alignment boundary, providing preliminary feature-level support for the Universality Hypothesis. We further identify and qualitatively characterize this microscopic fraction (0.02%) of diverging features, finding patterns consistent with the hypothesis that the recurrent bottleneck selectively limits the parsing of rigid syntax rather than broad semantic ontology. We demonstrate that while Pythia's unconstrained attention permits the monosemantic decomposition of distinct formatting edge-cases, Mamba is forced to compress unrelated syntactical anomalies into polysemantic "junk drawer" neurons to preserve state capacity. Collectively, these results suggest that architectural routing mechanisms may have negligible impact on core semantic understanding, with representational divergence confined to extreme structural margins.

0.07

Generalization in Neural Networks Through the Lens of Magnitude Potential

cs.LG ·2026-10-01 ·Sahel Torkamani, Henry Gouk, Rik Sarkar
similarity 0.60
Abstract

Explaining generalization and training dynamics in neural networks remains a challenge, and various approaches have been developed to study different aspects of these phenomena. In this paper, we introduce the idea of {\em magnitude potential} -- a quantity based on the theory of metric magnitude -- that reflects how well an arbitrary point is represented by a given set. We find that this basic quantity can be applied to examine various features in neural generalization. The ratio between the magnitude potential with respect to a class and with respect to the entire data, computed at the logit layer, is informative of the representation of the point. In experiments, these ratios for individual training points are found to be correlated with the Feldman memorization scores. Magnitude potential ratios aggregated across points detect structural changes in the decision boundaries and provide a geometric indicator of grokking in modular arithmetic. Although the magnitude potential ratio and neural collapse are both closely associated with intra-class and inter-class geometric structure, the magnitude potential ratio remains informative even when neural collapse is explicitly suppressed.

0.23

Beyond $L_2$: Generalizing Abductive Latent Explanations to Diverse Prototype-Based Architectures

cs.LG ·2026-08-17 ·Jules Soria, Alban Grastien, Romain Xu-Darme, Julien Girard-Satabin, Zakaria Chihani +1 more
published similarity 0.59
Abstract · Accepted at ECML-PKDD 2026, Research Track

Prototype-based neural networks are hailed as interpretable-by-design architectures. Recently, Abductive Latent Explanations (ALE) were introduced to provide formal, mathematically guaranteed explanations that leverage the intrinsic structure of these networks to ensure both predictive safety and human readability. ALEs rely on computing tight bounds on latent space distances to produce formal explanations. However, existing ALE formulations are rigidly confined to Euclidean latent spaces. This leaves a critical gap: modern state-of-the-art architectures increasingly rely on non-Euclidean representations - such as spherical metrics, Gaussian densities, and dimensional projections - rendering current formal explanation methods incompatible. In this work, we generalize the ALE framework to support non-Euclidean prototype architectures. For each geometric variant, we systematically derive how to either map the architecture to existing bounds or construct novel, architecture-specific bounding algorithms. We validate our theoretical constructions by computing subset-minimal formal explanations on fully trained image classifiers. By unifying these diverse models under a single formal framework, we enable the first rigorous, cross-architecture comparison of their interpretability.

0.21

SPICE: Simple Polysemantic Feature Interpretation via Clustering-based Explanation

cs.LG cs.CV ·2026-08-14 ·Sehyun Lee, Dahee Kwon, Damin Lee, Jaesik Choi
top venue similarity 0.58
Abstract · European Conference on Computer Vision -- ECCV 2026

One of the pivotal recent challenges in neural network interpretability is polysemanticity, where a single neuron is activated by multiple, often unrelated concepts, hindering clear functional understanding. Although prior work has explored this phenomenon, existing approaches remain architecture-specific and depend on manual heuristics such as a fixed number of concept clusters ($K$), limiting their generality and scalability--especially for modern Transformer-based models. To address these limitations, we introduce SPICE (\textbf{S}imple \textbf{P}olysemantic Feature \textbf{I}nterpretation via \textbf{C}lustering-based \textbf{E}xplanation), a generalizable framework for analyzing polysemanticity in deep vision architectures. SPICE avoids architecture-dependent propagation rules, enabling the first systematic comparison of polysemanticity across both CNNs and Transformers, and automatically determines the number of concept clusters per neuron, eliminating reliance on a preset $K$ and supporting scalable analysis for large models. Using SPICE, we conduct a comprehensive investigation into how polysemanticity emerges, varies across depth and architecture, and forms through distinct computational pathways.

0.40

Proving and Improving the Platonic Representation Hypothesis with JEPAs

cs.AI cs.CVstat.ML ·2026-10-29 ·Randall Balestriero, Yann LeCun
benchmarktheoryhighly citedcommunity pick similarity 0.58
Abstract

Whether models trained on different modalities converge to the same representation of the world is central to AI. The Platonic Representation Hypothesis (PRH) conjectures that they do as they scale, but without a proof it offers neither a guarantee nor a recipe. We turn the hypothesis into a theorem. First, we prove a \emph{Platonic Representation Theorem}: Joint-Embedding Predictive Architectures (JEPAs) trained separately on any number of modalities learn the same representation up to one rotation whenever their positive pairs are two observations of one scene--\emph{good views}--and their misalignment is bounded otherwise. Second, we show that next-token prediction is a JEPA with \emph{bad views}, parts of a single document, which explains why language models align with vision only through scale. Third, we validate the theory from 145k captions to the texts of 940M web images, and uncover a scaling law of alignment that predicts unseen runs within seed noise. Good views need 42 times fewer training samples than next-token prediction on the same corpus to reach the same alignment, and a 109M-parameter text JEPA aligns with DINOv2 and CLIP's image encoders better than every language model we evaluated, up to 72B parameters, and than CLIP's text encoder: next-token prediction is a remarkably inefficient route to platonic representations. As a showcase, one rotation turns separately trained encoders into an image captioner (CIDEr .838 on COCO). Even that rotation needs no pair: on the latest pair-free benchmark, our text JEPA beats every zero-shot accuracy reported with language models up to 8B parameters, e.g., 64\% against at most 56\% top-5 on ImageNet-100. Positive views, and not scale alone, are thus the lever to build platonic representations.

0.42

Neural Expansion: A Unified Mechanism for How Deep Neural Networks Generalize

cs.LG cs.AI ·2026-10-16 ·Chashi Mahiul Islam, Samuel Jacob Chacko, Mao Nishino, Canlin Zhang, Xiuwen Liu
top venuecode/data similarity 0.58
Abstract · Accepted at NeurIPS 2026. Includes supplementary material. Code: https://github.com/meetchashi/neural_expansion_neurips_2026

Deep neural networks demonstrate remarkable generalization despite being highly over-parameterized, challenging traditional statistical learning theories. While several theoretical paradigms such as margin-based bounds, PAC-Bayesian analyses, and neural tangent kernels provide global explanations, they often lack mechanistic insight into how generalization emerges from training dynamics. We introduce Neural Expansion, a unified geometric framework that characterizes generalization through the expansion of directionally-selective ray coverage in the input space. Using Generalization Intervals (GIs), we quantify direction-specific regions around training samples where the model's predictions remain stable. Our extensive empirical studies across diverse architectures and datasets reveal that training leads to progressive expansion of ray coverage, particularly along directions with low Jacobian singular values. This mechanism helps explain a large fraction of correctly classified test inputs, adversarial examples, and even mislabeled or out-of-distribution samples through the geometry of ray coverage rather than instance-level memorization. By connecting input-space curvature to predictive behavior, our framework provides a mechanistic foundation that complements existing global theories of deep learning.

0.58