CS Paper Radar

Differential Functionalism: Jacobians Shape the Structure of Phenomenal Experience

David Balduzzi

cs.AI cs.NE ·submitted 2026-09-08 18:01 UTC · v3
code/data

Abstract

This paper investigates the hypothesis that the first-order structure of physical interactions (Jacobians) shapes the structure of phenomenal experience. Two measures of Jacobian structure are introduced: transparency, the effective rank, which counts the independent differences that make a difference; and integrity, the normalised algebraic connectivity, which measures how hard a system is to split. Applied to a series of worked examples, the framework yields correlates of: (1) experiences that overlap without systems merging; (2) the boundaries of a system, which are selected by focus and limited by noise; (3) temporal thickness, the persistence of the past into the present, lengthened by a spread of timescales; (4) spatial structure under convolution, where unity at scale needs long-range connections; (5) the distinction between what is vivid and what is obscure, for errors and attention heads; (6) the incoherence of a randomly initialised network, as a structural model of James's "blooming, buzzing confusion"; (7) distinct versus confused ideas; (8) learning as accumulated experience; and, conjecturally, (9) the contrast between memory as duration in brains and as storage in computer hardware. The measures can be computed at scale in artificial networks and estimated from perturbational data in brains.

Comment
Code: https://github.com/dbalduzzi/diff_func; v2: substantially revised and retitled; spanning-tree cohesion replaced by integrity; new results

Closest papers

Nearest neighbours by meaning across the whole index.

Short Horizons and Sparse Concepts: a Mathematical View of the Readout in the J-lens

cs.CL ·2026-08-26 ·Shi-Qi Yan, Kai-Xuan Ding, Chao-Hong Tan, Qian Chen, Wen Wang +2 more
similarity 0.61
Abstract

The Jacobian lens (J-lens) has been proposed as a way to read verbalizable representations from language models. However, its principle and meaning lack a detailed and theoretical discussion. We provide a mathematical view of this interpretation and of its assumed causal structure. Besides treating the J-lens as a heuristic probe, we further regard it as a first-order causal transfer operator from intermediate activations to expected future readouts. We study the Jacobian matrix as the optimal local linear approximation of the downstream mapping, analyze its global approximation behavior and bias, and identify its mathematical meaning as an expectation over anticipated future readouts. Further analysis of the Jacobian energy distribution reveals that its causal geometry is highly sparse. The energy decays with depth, concentrates in an extremely small proportion, and decomposes into diagonal pathways and specific critical positions. This decomposition further resolves the expectation of the J-lens over future outputs into short-horizon and sparse concept predictions, providing a more intuitive attribution and explanation for the ability of the J-lens to visualize concepts during the thinking process. Based on the theory, we propose a simple but effective improvement strategy and decoupling method for the J-lens, which significantly enhances the ability of the J-lens to read out correct intermediate concepts.

0.43

Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks

cs.LG q-bio.NC ·2026-09-30 ·Lukas Braun, Erin Grant, Andrew M. Saxe
top venuehighly cited similarity 0.52
Abstract · 28 pages, 5 figures. ICML 2025 camera-ready version. Presented at ICML 2025 and CCN 2025

A foundational principle of connectionism is that perception, action, and cognition emerge from parallel computations among simple, interconnected units that generate and rely on neural representations. Accordingly, researchers employ multivariate pattern analysis to decode and compare the neural codes of artificial and biological networks, aiming to uncover their functions. However, there is limited analytical understanding of how a network's representation and function relate, despite this being essential to any quantitative notion of underlying function or functional similarity. We address this question using analysable two-layer linear networks and numerical simulations in non-linear networks. We find that function and representation are dissociated, allowing representational similarity without functional similarity and vice versa. Further, we show that neither robustness to input noise nor the level of generalization error constrain representations to the task. In contrast, networks robust to parameter noise have limited representational flexibility and must employ task-specific representations. Our findings suggest that representational alignment reflects computational advantages beyond functional alignment alone, with significant implications for interpreting and comparing the representations of connectionist systems.

0.72

Riding the Fixed-Point Curve: Image Classification in the Continuous Thought Machine

cs.LG cs.CVcs.NE ·2026-10-13 ·Thomas Hannagan
publishedtheory similarity 0.52
Abstract · 19 pages, 8 figures, 4 tables. Accepted at the NeurIPS 2026 Workshop on Symmetry and Geometry in Neural Representations (NeurReps), extended…

The Continuous Thought Machine (CTM) is a recurrent network that attends to inputs and makes predictions based on the synchronization between its neurons. On tasks spanning many domains, it displays emergent thought processes that have not been explained. We report a number of findings about how the CTM classifies images, focusing on ImageNet. Thinking steps are attracting but strongly non-normal maps exhibiting balanced amplification (Murphy and Miller, 2009); while attraction is already present at CTM initialization, training highly promotes non-normality. By Banach's theorem, the neural state thus has a locally unique attracting fixed point, which moves at every step, and the thought trajectory is the curve these fixed points trace, followed one tick behind. Most of that curve's motion is shared across images, and visible as a previously reported traveling wave of activity across neurons. The attention schedule along the thought is largely generic: shuffling queries across images at every tick costs only 7 percent of classification accuracy, while image-independent increments predict the next query with R^2 = 0.98 (held out). Our results suggest that the CTM has found a single strategy to process all ImageNet images.

0.36

A Mathematical Theory of the Jacobian Lens

cs.LG cs.AIq-bio.NC ·2026-10-22 ·Mitchell Ostrow, Daoyuan Qian, Toni Liu, Ila Fiete
similarity 0.52
Abstract · 27 pages, 19 figures

Recent work from Anthropic introduced a new interpretability tool, called the Jacobian lens (J-lens, Gurnee et al., 2026). This method identified linear directions in the hidden states of frontier language models that had properties reminiscent of working memory-like representations. However, it is unclear what mechanistic properties of such features make them detectable to the J-lens. To this end, we introduce a mathematical theory of the J-lens, and show that it preferentially identifies coherent features -- directions that act like leading, spatially coherent Lyapunov modes. Small perturbations along these directions amplify across layers, remain roughly constant across token positions, and are consistent across different contexts. While we are agnostic to whether these properties are necessarily associated with a ``Global Workspace'', the analytically derived preferences of the J-lens nevertheless explain prior empirical applied to both model classes. We validate our analysis on a Bayes filter and a toy transformer, each optimally predicting next tokens emitted from a Hidden Markov Model (HMM). We show how modulating task parameters that change the coherence of features in the Bayes filter correspondingly and predictively modulates the response of the J-lens . We also introduce of a family of J-lenses built to emphasize different temporal scales of information processing in LLMs, which we call $ω$-J-lenses.

0.22

When Models Don't Manipulate Manifolds: The Geometry of a Comparison Task

cs.LG cs.AIcs.CL ·2026-09-29 ·Sai Sumedh R. Hindupur, Hadas Orgad, Thomas Fel, Demba Ba
similarity 0.51
Abstract

One of the current premises of mechanistic interpretability research is that detailed accounts of the geometry of neural network representations can tell us how models perform computations, and how to effectively intervene on them. While low dimensional manifolds have been observed for multiple concepts in the literature (e.g. numbers encoded on helices, days of the week on a circle, ...), with structure believed to reflect properties of data and tasks, the extent to which models rely on them for computation, and how they manipulate them, remains unclear. We characterize precisely the geometry of computation in a number-comparison task, as an abstraction of comparison for decision making, and how models utilize geometry in an elegant fashion to implement it. Specifically, we study the causal geometry of number comparison in Qwen2.5-7B-Instruct, a capable and widely studied open-weight model, and find Qwen largely uses linear representations of numbers despite the presence of curved geometry. To compare two numbers, the model first encodes each number along a vector and adds the two representations using attention and the residual connection, bringing them into a shared space in the residual stream. Then, the model uses MLP neurons to compare the pair of numbers on local regions in this shared space, which correspond to smaller intervals of input numbers, and combines these to obtain the position of the maximum. In fact, this reliance on linear representations for comparison also persists when the model compares three numbers. Our findings demonstrate that the manifold hypothesis can co-exist with linear representations: while concepts that are ordered may have manifold structure in representations, the model may use an underlying linear structure of the concept in certain computations.

0.26

Learning Perceptual Rules in a Fragmented Visual World

eess.IV ·2026-09-26 ·Tianqin Li, Isaac Lin, Leiran Song, Tai Sing Lee
hot direction similarity 0.50
Abstract

Visual perception organizes incomplete sensory evidence into coherent structures, but where do the principles governing this organization come from? We investigate whether learning to predict missing visual information can give rise to structural priors resembling classical Gestalt principles. Masked autoencoders (MAEs) trained on natural images exhibit perceptual completion consistent with good continuation, symmetry, and global form. Critically, when local evidence surrounding a missing region is held constant, manipulating distant elements systematically changes its reconstruction, demonstrating that local predictions are constrained by the global organization of the visual input. We next manipulate visual experience by adapting MAEs to either natural images or texforms that preserve local image statistics while disrupting global spatial organization. This altered experience weakens globally supported completion and shifts perceptual inference toward local continuation when local and global constraints conflict. Together, these results suggest that perceptual organization reflects structural regularities of natural scenes acquired through visual prediction: predictions integrate structural priors across multiple spatial scales, and the balance between local and global organization adapts to the statistical structure of visual experience. More broadly, our findings suggest that classical Gestalt principles may emerge as adaptive perceptual priors learned from the statistical structure of the visual world.

0.59

A Spectral Theory of Compositional Learning

cs.LG ·2026-09-27 ·Hugo Rydel
similarity 0.50
Abstract · 22 pages, 9 figures, and 2 tables. Under review at ICLR 2027

How does compositional reasoning emerge during learning? We address this question by mathematically analyzing the learning dynamics of deep linear networks. We train these networks in structured synthetic environments and derive a theory linking the structure of experience to compositional learning. Our theory predicts when compositional inferences emerge, whether they are identifiable from the available evidence, and how new linking evidence can rapidly unlock previously unavailable inferences. These results provide a qualitative explanation for several phenomena observed in human cognition. They account for why a composition can fail despite knowing its premises, why similar compositions can emerge at different times, and how a single linking fact can suddenly enable many new inferences. Taken together, these findings establish a mathematical link between the statistical structure of experience and the development of compositional reasoning.

0.24

Pixels over Symbols: Sensory Realism Improves Behavioral Alignment in Models of Cognition

cs.AI q-bio.NC ·2026-11-07 ·Hao Yuan Bai, Xiaoxuan Lei, Zihan Weng, Motahareh Pourrahimi, Lucas Gomez +1 more
similarity 0.50
Abstract

Computational models of cognition have traditionally relied on abstract symbolic inputs that approximate sensory experiences with simplified vector representations. This simplification reflects historical computational constraints and a long-held assumption that the details of sensory realism are incidental to the study of cognitive processes. Recent advances have enabled the development of sensory-realistic models capable of performing diverse cognitive tasks, yet whether sensory realism meaningfully shapes cognition has not been formally established. We address this gap through a large-scale empirical comparison of abstract and sensory-realistic neural network models trained on a battery of memory-dependent decision-making tasks. Our results show that sensory-realistic models mirror human behavioral response patterns significantly more than their abstract counterparts, though a substantial gap to human internal behavioral consistency remains even for the best tested models. By analyzing the internal dynamics of these systems, we find that the learned recurrent weights are not the main source of this divergence. In fact, frozen abstract-trained weights driven by sensory-realistic inputs recover the human consistency of sensory-realistic models. The resulting dynamics are also within the seed-to-seed variability of sensory-realistic models. Together, our findings argue that across a broad set of visual working-memory tasks, sensory realism accounts for a substantial portion of the model-human behavioral agreement, more than the choice of sequence architecture and comparable to the choice of vision encoder. These results suggest that findings obtained from abstract task models in neuroscience should be more carefully interpreted in light of their potential behavioral limitations.

0.17