CS Paper Radar

Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs

Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis, Mikhail Belkin

cs.LG stat.ML ·submitted 2026-08-25 02:54 UTC · v3

Abstract

Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has focused on the emergence of a global low-dimensional representation. We show that this picture is incomplete. In regression problems with clustered data, we demonstrate that multilayer perceptrons (MLPs) develop specialized neurons: individual neurons become strongly aligned with specific features relevant to a particular region of the input space. A substantial fraction of these neurons exhibit monosemanticity: they activate predominantly on inputs from the corresponding cluster. Thus, rather than learning a single global low-dimensional representation, MLPs learn a collection of local low-dimensional representations. Furthermore, we show that this ability to specialize gives MLPs a provable data-efficiency advantage over feature-learning methods based on a global low-dimensional representation.

Closest papers

Nearest neighbours by meaning across the whole index.

A "feature ODE" describing the learning behavior of shallow MLPs on simple functions

cs.LG cs.AI ·2026-10-19 ·Joseph Turnbull, Berkan Ottlik, Jamie Simon
published similarity 0.63
Abstract · Accepted as part of the HiLD workshop, ICML 2026

We study the gradient flow training dynamics of two-layer multi-layer perceptrons (MLPs) on anisotropic Gaussian data. Even in this simple setting, solving the dynamics in full is notoriously difficult, so we try something different. We propose a "feature ODE" that might be intuitively expected to describe the learning dynamics of a shallow MLP. We find that MLPs trained to learn simple target functions (low-order Hermite polynomials and staircase functions) actually roughly follow the loss trajectories from this feature ODE. This agreement is a surprise because of the size of the simplification and the fact that it does not come from a rigorous chain of approximations. We discuss this surprise and speculate on its reasons and uses.

0.40

The Tangled Web They Weave: Exploring Neural Representations with Directed Simplicial Complexes

cs.LG ·2026-10-31 ·Michael Bleher
published similarity 0.63
Abstract · 52 pages, 16 figures. Accepted at NeurReps 2026 (Symmetry and Geometry in Neural Representations)

Understanding how artificial neural networks (ANNs) represent and combine features internally is a central challenge in mechanistic interpretability. Existing neural circuit-discovery methods represent neuron interactions as pairwise edges in a graph. But distributed neural representations consist of large groups of neurons that are inherently linked by higher-order causal dependencies: suppressing a subgroup of neurons in concert may shift the activation of other neurons in the feature representation. We propose a method to construct filtered directed simplicial complexes (DSCs) that give a topological representation of such dependencies, built from the activation profiles of an ANN on a given set of inputs. Our approach treats ANNs as structural causal models and uses counterfactual ablations to test for causal links between groups of internal neurons within specific input contexts. As a proof of concept, we apply the approach to a pair of small multilayer perceptron (MLP) architectures in a synthetic two-dimensional quadrant classification task. In these examples, DSCs recover the designed feature pathways and coordinated class transitions, and the complexes of trained networks are higher-dimensional than those of their handcrafted references. As validation, we compare our method with graph-based circuit-discovery baselines and find that their 1-skeletons agree within the relations that a weight graph can represent. These results support DSCs as context-dependent descriptions of multi-neuron computation and may thus provide a novel, orthogonal framework to study the phenomenon of distributed feature representations and polysemanticity.

0.33

Parameter Symmetries Determine Representational Geometry in Overparameterized Nonlinear Networks

cs.LG ·2026-09-30 ·Marvin Theiss, Lukas Braun, Andrew M. Saxe, Erin Grant
top venuecode/data similarity 0.63
Abstract · To appear in NeurIPS 2026. 79 pages, 10 figures. Code: https://github.com/mrvnthss/symmetries-representational-geometry

Representations are routinely used across machine learning, psychology, and neuroscience to draw inferences about the computations of biological and artificial systems. Such inferences presume that the geometry of these representations reflects the computation being performed. For artificial neural networks, however, the extent to which the function a network computes constrains this geometry remains unclear. One key obstacle is that these networks admit parameter symmetries: changes in parameterization that preserve function exactly while reshaping representational geometry. Here, we show that a broad class of parameter symmetries acts on representations through just three primitive feature transformations: addition, duplication, and scaling. This feature-level characterization yields a closed-form decomposition of representational geometry into essential and auxiliary components, revealing how representational degeneracy can exist without and grows with overparameterization even when function is held fixed. Finally, we establish conditions under which implementation-level selection rules resolve this degeneracy, yielding identifiable geometries in which features are weighted according to their contributions to the network's function. Together, our results delineate when representational geometry can support inferences about computation, and when it cannot.

0.56

Mesoscopic learning dynamics of extensive-width neural networks

cs.LG stat.MLcond-mat.dis-nncond-mat.stat-mech ·2026-10-16 ·Shu Zhou, Jean Barbier, Sebastian Goldt
similarity 0.62
Abstract

Understanding how stochastic gradient descent learns internal representations in wide neural networks trained on rich target functions is a central problem in machine learning. We study this question in two-layer neural networks learning target functions represented by networks of the same architecture, with arbitrary nonlinearity, and with network width and input dimension growing proportionally. In this regime, standard approaches for describing the learning dynamics require tracking a diverging number of overlaps, which measure correlations between learned and target features, leaving no obvious description in terms of a few order parameters. Here, we develop a mesoscopic description of learning by characterizing the distribution of overlaps. In the early stages of learning, we find that a Gaussian description of overlap statistics reduces the learning dynamics to a few collective variables that track learning of the target function's linear and quadratic components. We then estimate the timescale of specialization, when Gaussian closure breaks down and individual learned features begin to strongly correlate with specific target features. We find that specialization in extensive-rank networks is accelerated via a quadratic shielding mechanism, whereby learning the linear and quadratic components of the target function filters gradient fluctuations. We finally describe how the activation function determines the pattern of specialization and characterize the subsequent refinement of learned features. Together, these results show how collective learning gives way to individual feature discovery and clarify when learning in large neural networks admits a simple statistical description.

0.32

Learning Computational Structure for Sample-Efficient Prediction

cs.LG cs.AI ·2026-10-20 ·Hangyu Zhou, Constantine Dovrolis, Stephen Mussmann
benchmark similarity 0.61
Abstract · 36 pages

Statistical learning in high dimensions suffers from the curse of dimensionality. Fortunately, many functions of practical interest are compositionally sparse, meaning they admit a composition of local functions, each depending on only a few variables, allowing sample complexity to depend on local arity rather than ambient dimension. Yet realizing this benefit requires exploiting hidden compositional structure, which theory does not establish for practical gradient-based training. We study this gap with CSFRBench, a controlled benchmark that hides the target's compositional computation during training but retains it for diagnosis, and find that dense networks and sparsity-based baselines leave much of that benefit unrealized. Inspired by the spatial organization of biological neural circuits, we introduce Learned Wiring (LeWi): each neuron has a learned low-dimensional embedding, only nearby neurons connect, and embeddings and weights are trained jointly. LeWi improves sample efficiency on targets with hierarchical structure, intermediate reuse, or independent modules. Across 32 additional targets generated by independently sampled computation graphs, LeWi is the most sample-efficient method not given the target graph, with 27% lower mean learning-curve area than dense networks. Dense networks perform better on targets with a single high-arity function. LeWi's representations and connectivity align with the target computation. On 17 world-model prediction tasks spanning control, navigation, and manipulation, LeWi beats the dense baseline on 16 and wins 99/119 baseline comparisons. These results show that jointly learning connectivity and weights can exploit computational structure to improve sample efficiency.

0.21

Removing spurious minima for planar features by skip connections

cs.LG cs.AImath.OC ·2026-10-01 ·Jakob Paul Zimmermann, Andrei Balakin, Moritz Grillo, Georg Loho
code/data similarity 0.61
Abstract · 43 pages, 4 figures. Under review. Accompanying Lean 4 formalization available at https://github.com/JayPiZimmermann/Removing-spurious-minim…

Understanding loss landscapes is central to explaining neural-network training, yet their structure remains only partially understood even in simple models. We study the Gaussian population loss of shallow, bias-free ReLU networks in the teacher--student setting. This provides a simple model for studying essential aspects such as feature learning and overparameterization. For teacher networks with positive output weights and planar features, we show that including a learned linear skip removes all spurious local minima with non-negative student output weights once the student network is at least as wide as the teacher network. In contrast, without the skip, we construct a fixed teacher network with positive output weights and only three hidden neurons in input dimension two whose spurious local minima persist at every student width at least three. Thus, a learned linear skip can remove spurious minima that persist under arbitrary overparameterization. Furthermore, we show that a positive output weight student network always learns the subspace spanned by the teacher features: student features at local minima with non-negative student output weights lie in the span of the teacher features. For ReLU networks in two dimensions, even heavily overparameterized student networks have effective width controlled by the teacher width: every critical point with positive student output weights has at most twice as many distinct student feature directions as teacher neurons. Finally, we transfer the benignity result to empirical minima over parameter balls of any prescribed radius, with the required sampling accuracy depending on that radius.

0.42

Sparsity-driven emergence of functional modularity in multitask neural networks

cs.LG cs.AI ·2026-10-23 ·Bowen Shi, Qi Ouyang, Long Qian
similarity 0.61
Abstract · 34 pages, 6 figures, including Supplementary Information

Neural networks can learn multiple tasks, but how learning organizes shared computation into specialized and reusable components remains poorly understood. In this work, we show that sparsity inducing regularization promotes functional modularity in multitask networks without predefined module boundaries. Using functional profiling in multilayer perceptrons and a lightweight transformer, we identified compact, task-selective subnetworks, termed functional tokens. Across rule-based task families, selectively retaining these tokens preserves their associated functions while reducing performance on other tasks. Such specialization develops preferentially in deeper layers during training, while earlier layers retain shared computation. Regularization strength and form shape module compactness and specialization, while task weighting alters functional allocation. Moreover, extracted functional tokens can be recombined through alternative pathways to reconstruct a learned composite computation without retraining, preserving their connectivity and learned weights. Our results suggest that sparsity-constrained optimization shapes not only parameter use but also the organization and reuse of learned computation.

0.18

Do Neural Models Prefer Their Submodels?

cs.LG math.AGstat.ML ·2026-10-20 ·Alvaro Ribot, Yulia Alexandr, Anna Seigal
similarity 0.61
Abstract · 29 pages, 7 figures

A neural model is the set of functions that can be expressed by a given neural network architecture. Neural models contain natural submodels, such as those obtained by setting some neurons to be inactive. We study the geometry of neural models and ask whether they prefer their submodels. By this we mean whether local minimizers of the loss function lie in a given submodel. We introduce a geometric framework to determine if this occurs. For shallow multilayer perceptrons, we show that subnetworks with inactive hidden neurons are never preferred. For deep multilayer perceptrons, we give conditions under which subnetworks with inactive hidden neurons are not preferred. For lightning self-attention, lower-rank submodels are usually not preferred, but we identify examples where they are because they lie on a boundary of the model. Our results show that, aside from a few exceptions, preference for submodels cannot be explained by the geometry of the neural model, and must instead come from other sources such as training dynamics.

0.56