CS Paper Radar

The Tangled Web They Weave: Exploring Neural Representations with Directed Simplicial Complexes

Michael Bleher

cs.LG ·submitted 2026-10-31 09:58 UTC · v1
published

Abstract

Understanding how artificial neural networks (ANNs) represent and combine features internally is a central challenge in mechanistic interpretability. Existing neural circuit-discovery methods represent neuron interactions as pairwise edges in a graph. But distributed neural representations consist of large groups of neurons that are inherently linked by higher-order causal dependencies: suppressing a subgroup of neurons in concert may shift the activation of other neurons in the feature representation. We propose a method to construct filtered directed simplicial complexes (DSCs) that give a topological representation of such dependencies, built from the activation profiles of an ANN on a given set of inputs. Our approach treats ANNs as structural causal models and uses counterfactual ablations to test for causal links between groups of internal neurons within specific input contexts. As a proof of concept, we apply the approach to a pair of small multilayer perceptron (MLP) architectures in a synthetic two-dimensional quadrant classification task. In these examples, DSCs recover the designed feature pathways and coordinated class transitions, and the complexes of trained networks are higher-dimensional than those of their handcrafted references. As validation, we compare our method with graph-based circuit-discovery baselines and find that their 1-skeletons agree within the relations that a weight graph can represent. These results support DSCs as context-dependent descriptions of multi-neuron computation and may thus provide a novel, orthogonal framework to study the phenomenon of distributed feature representations and polysemanticity.

Comment
52 pages, 16 figures. Accepted at NeurReps 2026 (Symmetry and Geometry in Neural Representations)

Closest papers

Nearest neighbours by meaning across the whole index.

ReGraph: A Computational Account of Emergent Generalization in the "what" and "where" Dual Visual Streams

cs.NE cs.AIq-bio.NC ·2026-10-06 ·Hyewon Kang, Jungmin Lee, Ilgyu Lee, Seok-Jun Hong
benchmark similarity 0.68
Abstract · 29 pages, 6 figures

Where generalization capacity--the ability to extract context-invariant relational structures--first emerges remains a central question in AI and neuroscience. The foundation for this capacity lies upstream of the hippocampus, within the entorhinal cortex, where parallel pathways dissociate relational structure in the medial entorhinal cortex (MEC) from sensory content in the lateral entorhinal cortex. However, as Eichenbaum argued, such factorization likely originates earlier, driven by the segregation of the dorsal ('where') and ventral ('what') visual streams. Supporting this, grid-like firing patterns--a signature of MEC (context-invariant codes)--also appear in preceding neocortical regions along the dorsal pathway. Yet, how such representations are computationally formed along upstream pathways remains unknown. To investigate this in silico, we developed ReGraph, a recurrent dual-stream graph model with biological inductive biases, including retina-driven stream-specialized encoding, dorsal-to-ventral modulation, and dynamic lateral connectivity. Trained on the action benchmark Something-Something V2, ReGraph revealed a pathway-specific emergence of relational mapping: context-invariant codes and grid-like spatial bases uniquely co-emerged along the extended dorsal stream. In contrast, their absence in single-stream, unmodulated variants, and standard baselines implies that these inductive biases are prerequisites for relational structures. Crucially, our post-hoc analyses demonstrated that these grid-like bases serve as reusable routing templates for information processing via lateral connectivity. Together, our findings provide a computational account that generalization may not be a faculty that emerges abruptly within a dedicated region, but a property that already takes shape as sensory information is parsed into factorized streams of hierarchical visual processing.

0.32

Mechanistic Interpretability Needs a Logic of Inference

cs.CL cs.CV ·2026-11-10 ·Zhimin Hu, Chen Jiang, Hanbo Xie, N. Apurva Ratan Murty, Sashank Varma
top venuehot direction similarity 0.65
Abstract · NeurIPS 2026 Interp4Discovery (Oral)

Neural networks are increasingly studied not merely as predictive systems, but as complex systems whose internal computations are themselves objects of scientific inquiry. Recent progress in mechanistic interpretability (MI) has made it possible to identify neurons, features, circuits, population structures, and dynamical patterns associated with model behavior. As these tools improve, attention is shifting from what can be observed inside a model to how those observations support mechanistic explanations. We examine the parallel efforts of neuroscience and MI to address this question and identify recurring inferential bottlenecks when inferring mechanisms from complex neural systems. We propose five principles and organize them into a recursive logic for a more rigorous procedure of mechanistic inference. These principles provide a framework that supports scientific discovery in two senses: it helps turn local interpretability findings into cumulative knowledge of mechanisms within models, and it makes those mechanisms sufficiently precise to generate testable hypotheses about domains beyond the models. Evidence beyond MI is ultimately required to establish what model mechanisms reveal about the world.

0.37

Mechanistic Interpretability of Deep Neural Networks: Methods, Frameworks, and Perspectives

cs.LG cs.AIcs.CV ·2026-11-10 ·Lorenzo Cirillo, Irene Amerini, Andrea Cavallaro
survey similarity 0.64
Abstract

Deep neural networks underpin fundamental tasks in a wide range of signal domains, such as image, audio, and language processing. However, their internal mechanisms are notably difficult to interpret. Internal activations are structured signals transformed across layers that may encode a multitude of interdependent relationships, as individual neurons frequently represent multiple underlying concepts. Mechanistic interpretability aims to uncover the internal mechanisms of neural networks by identifying components whose activations are causally responsible for specific behaviors. In such a context, this article provides an overview of advances in mechanistic interpretability for the analysis of causal inference in deep neural networks.

0.29

Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs

cs.LG stat.ML ·2026-08-25 ·Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis, Mikhail Belkin
similarity 0.63
Abstract

Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has focused on the emergence of a global low-dimensional representation. We show that this picture is incomplete. In regression problems with clustered data, we demonstrate that multilayer perceptrons (MLPs) develop specialized neurons: individual neurons become strongly aligned with specific features relevant to a particular region of the input space. A substantial fraction of these neurons exhibit monosemanticity: they activate predominantly on inputs from the corresponding cluster. Thus, rather than learning a single global low-dimensional representation, MLPs learn a collection of local low-dimensional representations. Furthermore, we show that this ability to specialize gives MLPs a provable data-efficiency advantage over feature-learning methods based on a global low-dimensional representation.

0.21

Connectome-to-Function: Conditional Generative Latent Representations for Reservoir Computing

cs.LG ·2026-09-05 ·Zhuolin Yu, Xingyu Liu, Yuanhao Jia, Yunhang Xiao, Hairuo Xue +2 more
similarity 0.62
Abstract · 9 pages, 4 figures. Supplementary material included

Connectomes, graph-level maps of neurons and their synaptic connections, provide a structural basis for understanding how brain circuits support function and computation. However, mapping connectome structure to computation remains difficult because these graphs are high-dimensional, sparse, and sensitive to local structural variation. Existing approaches often depend on hand-crafted structural descriptors or task-specific predictors, which limits their ability to represent connectomes in a form that is both generative and functionally meaningful. We propose a conditional generative latent framework that encodes connectome graphs into a compact structural space while using available node-level conditions to guide reconstruction and generation. From this space, the model can reconstruct observed connectivity with a mean edge-reconstruction AUC up to 0.910 and generate new candidate connectomes, enabling a unified analysis of graph structure and computational behavior. Using connectome-derived graphs as recurrent computational substrates, we found that the learned latent space captures functional variation across reservoir-computing experiments, with cross-validated $R^2$ values up to approximately 0.87. Interpretability analysis further revealed task-specific structural mechanisms: in our examples, memory performance is associated with reciprocal recurrent connectivity, whereas prediction and classification are more strongly associated with spectral properties of the recurrent network. These findings suggest an AI-for-science approach to linking neural connectivity to computation and provide a generative and interpretable basis for studying how distinct structural mechanisms shape computational capacity.

0.27

Parameter Symmetries Determine Representational Geometry in Overparameterized Nonlinear Networks

cs.LG ·2026-09-30 ·Marvin Theiss, Lukas Braun, Andrew M. Saxe, Erin Grant
top venuecode/data similarity 0.62
Abstract · To appear in NeurIPS 2026. 79 pages, 10 figures. Code: https://github.com/mrvnthss/symmetries-representational-geometry

Representations are routinely used across machine learning, psychology, and neuroscience to draw inferences about the computations of biological and artificial systems. Such inferences presume that the geometry of these representations reflects the computation being performed. For artificial neural networks, however, the extent to which the function a network computes constrains this geometry remains unclear. One key obstacle is that these networks admit parameter symmetries: changes in parameterization that preserve function exactly while reshaping representational geometry. Here, we show that a broad class of parameter symmetries acts on representations through just three primitive feature transformations: addition, duplication, and scaling. This feature-level characterization yields a closed-form decomposition of representational geometry into essential and auxiliary components, revealing how representational degeneracy can exist without and grows with overparameterization even when function is held fixed. Finally, we establish conditions under which implementation-level selection rules resolve this degeneracy, yielding identifiable geometries in which features are weighted according to their contributions to the network's function. Together, our results delineate when representational geometry can support inferences about computation, and when it cannot.

0.56

Temporal Geometry of Deep Networks: Hyperbolic Representations of Training Dynamics for Intrinsic Explainability

cs.LG cs.AIcs.NEphysics.data-an ·2026-10-02 ·Ambarish Moharil
top venue similarity 0.61
Abstract · Published as a main conference paper at ICLR 2026. 10 Main Pages, 22 pages of supplementary material

Intrinsic explainability remains a challenging problem, particularly in contexts where multilayer perceptrons (MLPs) require dynamic re-training within an optimization environment. This paper investigates how MLPs and their training dynamics can be represented and studied in non-Euclidean spaces; our representation features the Poincaré model of hyperbolic geometry. We aim to capture the geometric evolution of their weighted topology and self-organization over time. Instead of restricting the analysis to single checkpoints---as per established measure-based explainability methods---we construct temporal \textit{parameter graphs}, i.e., snapshots over time $T$ steps of the optimization/training process for MLPs. This reflects the view that neural networks encode information not only in their weights but also in the trajectory traced during training. Drawing on the idea that many complex networks admit embeddings in hidden metric spaces where distances correspond to connection likelihood, we present a geometric and temporal graph-based metalearning framework for obtaining dynamic hyperbolic representations of the underlying neural parameter graphs. Our model embeds temporal parameter graphs in the Poincaré model ball, and learns from them while maintaining equivariance to within-snapshot neuron permutations and invariance to permutations of past snapshots. In doing so, the approach preserves functional equivalence over time and recovers the latent evolving geometry of the network. Experiments on regression and classification tasks with trained MLPs show strong meta-network performance, accompanied by hyperbolic temporal representations. This reveals how the network structure emerges over time under specific training environments, thus providing insights into the network's self-organization.

0.55

Graphlets as structural fingerprints of complex networks

cs.SI q-bio.QM ·2026-09-15 ·Anna Pidnebesna, David Hartman, Aneta Pokorna, Daniel Trlifaj, Jaroslav Hlinka
similarity 0.61
Abstract

Complex networks are often compared using selected graph-theoretical measures that capture a selected set of properties with effects ranging from local to global, such as degree, clustering or betweenness centrality. Here we introduce a structural fingerprinting framework based on graphlets: small rooted subgraphs whose distributions provide a systematic description of local-to-mesoscale topology. Across synthetic networks generated from several random graph models, graphlet fingerprints capture parameter-dependent structural differences, outperform standard graph-theoretical measures, and identify even subtle local patterns driving discrimination. We then apply the framework to empirical resting-state functional connectomes, documenting that while graphlets show superior sensitivity also to controlled topological perturbations of brain connectivity, specifically in schizophrenia-control classification they perform only comparably to classical graph-theoretical features. This is in line with the notion that schizophrenia-related alterations are dominated by spatially localized connectivity changes rather than general topological reorganization. Altogether, the generative modeling, targeted perturbations and real-world neuroimaging classification challenge position graphlets as flexible structural fingerprints of complex networks, while carefully outlining their strength and weaknesses compared to more classical graph theoretical features.

0.16