CS Paper Radar

Reading a neuron's tuning from its correlations with other neurons

Francesco Lässig

cs.LG q-bio.NCcs.NE ·submitted 2026-09-19 10:16 UTC · v1
code/data

Abstract

What a neuron represents is usually established by relating its activity to a stimulus. We ask whether the same facts are contained in the correlation matrix of a recorded population, which keeps the pairwise relations among the neurons and discards the stimulus. A transformer that sees only the correlation matrix of 512 sampled neurons, with no labels and no reference population in its input, decodes the preferred orientation of mouse visual cortex neurons from in-vivo MICrONS responses to a mean error of 25° (uninformed: 45°; best constant guess: 41°) and their receptive-field position at R^2 = 0.23; on the digital-twin substrate, 20° and 0.48. Asked whether a neuron prefers horizontal or vertical, it is right for 79% of neurons in vivo and 87% on the twin, against 56% for the majority answer; it does not resolve the obliques. The ordering is recoverable because the relations between orientation classes are close to circulant, which fixes the class structure up to rotation and reflection; the rotation of the frame is recoverable only because of a departure from that symmetry. The departure survives equal representation of every orientation and disjoint training and test stimuli, and it is an anisotropy of the signal correlations: in V1 the neurons preferring the dominant orientation have the most similar responses to one another, and a model population with only that property recovers the frame. Across 33 mice of the Allen Brain Observatory, with relations from drifting gratings, orientation transfers to unseen animals only when a sampled population mixes animals; populations drawn from one animal are not decoded measurably below the best constant, a limit of the diversity of populations one animal provides. For receptive fields, what crosses animals is where each animal's imaged patch looks on the screen, not the layout of neurons within it.

Comment
Code: https://github.com/entropicbloom/intentionality. Reassignment to q-bio.NC as primary welcome

Closest papers

Nearest neighbours by meaning across the whole index.

Structure is in the zoom: probing neural symmetry through dimensionality scaling

cs.CL q-bio.NCcond-mat.dis-nncs.LG ·2026-09-16 ·Adil Amin
top venuebenchmarkcode/dataunusual similarity 0.62
Abstract · Accepted at NeurReps 2026 (Symmetry and Geometry in Neural Representations), Proceedings of Machine Learning Research. 48 pages. Code, data …

Scaling exponents of neural dimensionality are read as properties of the code, and used to compare representations and benchmark models against cortex. Yet one recording of 11,311 mouse V1 neurons gives 0.25, 0.30 or 0.35, depending only on how it is subsampled. We show that such an exponent measures the population and the probe together. We split it exactly into a floor, which random subsets of the same sizes give, and a shift, which a declared axis of stimulus classes adds. The usual random-subsample exponent is almost all floor. Stimulus direction lifts the exponent in every grating recording by reading both symmetry sectors of the code: orientation, which repeats every half turn, and direction, which does not. The lift lives in trial-to-trial fluctuations that differ between directions, while the class means, which linear decoders read, pull it down. Two measured terms, how much the classes' fluctuations overlap and how the means sit against them, account for the shift. They give its sign where it changes: in Allen Neuropixels recordings, whose class means mostly raise the exponent, and in macaque V1. In an XY lattice whose classes stay on one orbit, temperature flips the sign as the two terms say, and in GPT-2 the shift ranks the indirect-object circuit's name-mover and backup heads. Reported with its path, its floor and its shift, an exponent says how much structure a probe finds and what carries it. Our open tool, rung, reports all three for any labelled neural recording or language-model layer: https://github.com/adilamin89/structure-in-the-zoom

0.68

The Computational Value of Sensory-Aligned Receptive Fields Depends on Neuronal Expressivity

cs.NE cs.LGq-bio.NC ·2026-09-22 ·Agnese Adorante, Aaron Spieler, Anna Levina
similarity 0.52
Abstract

Biological sensory neurons have selective receptive fields organized along meaningful stimulus coordinates, such as frequency, motion direction, or retinotopic position. Such structure may arise from efficient coding and biological constraints on activity, connectivity, and wiring, as computational studies of simple neurons have shown across modalities. This raises a question: do structured receptive fields confer a computational advantage beyond resource efficiency itself, and does this advantage persist when individual neurons are highly expressive? We address this question in recurrent networks of Expressive Leaky Memory neurons, where we can independently vary neuronal complexity and the organization of feed-forward receptive fields. Across auditory and event-based visual classification tasks, receptive fields aligned with a task-relevant sensory coordinate improve test accuracy relative to budget-matched random receptive fields. This advantage disappears when sensory coordinates are scrambled, or when receptive fields follow task-irrelevant coordinates, showing that the benefit comes from alignment with task geometry rather than restricted connectivity alone. Increasing neuronal complexity reduces the performance advantage of structured receptive fields. Finally, generic synaptic sparsity regularization induces input selectivity and partially recovers performance, but remains substantially below explicitly structured receptive fields, suggesting that sparsity alone is insufficient to recover the full computational benefit of task-aligned receptive fields. Together, our results show that appropriate receptive fields can serve as a computational prior beyond sparsity itself, and that their value depends on the computational expressivity of individual neurons.

0.23

Evaluation Resolution Confounds Learning-Rule Comparisons in Model-Brain RSA of Early Visual Cortex

cs.LG q-bio.NC ·2026-08-11 ·Nils Leutenegger
similarity 0.51
Abstract · v3: RSA per subject on cross-run stimulus pairs; V1 gap +0.044 -> +0.030 [0.018, 0.042], still ~0 at 32 px. Leave-one-subject-out lower boun…

Studies of biologically plausible learning rules (feedback alignment, predictive coding, STDP) train small networks on $32\times32$ images and compare them with brain responses to naturalistic stimuli presented at much higher resolution. We find that a common result in this setting, that untrained or locally trained networks rival or beat backpropagation at early visual cortex, depends on the resolution at which networks are evaluated. In representational similarity analysis (RSA) against human V1, the gap between an untrained and a backprop-trained network grows monotonically from $-0.001$ (95% CI $[-0.011, 0.009]$) at the 32 px training resolution to $+0.030$ $[0.018, 0.042]$ at 224 px (six resolutions, 5 seeds; per-subject RSA on cross-run stimulus pairs). The growth reproduces for three further learning rules, directionally in single-seed macaque electrophysiology, along training, and for an ImageNet ResNet-50 and a Swin-Tiny transformer, which also align best at low resolution although trained at 224 px. Gabor and pixel structure, the normalization state of the untrained baseline, and a global brightness statistic do not account for it. Against a criterion fixed before the analysis, limiting image content to 32 px while the input still grows does not shrink the V1 gap ($+0.035$ $[0.023, 0.046]$ vs. $+0.030$ at 224 px; ratio 1.14 $[1.03, 1.34]$): it arises once the input exceeds the training resolution, through a decline of backprop. A single luminance value per image reaches $ρ= 0.054$ against V1, matching the untrained network (0.053), which bounds what RSA on globally pooled features can resolve at V1 in this dataset. The one learning effect that holds across resolution is backprop above untrained at LOC. Comparisons of learning rules or architectures at early visual cortex need to control, and report, the evaluation resolution.

0.34

Do Vision Model See Like the Brain? A Comparison Across EEG Encoding Model

cs.CV cs.AIcs.HC ·2026-09-22 ·Shashank Baghel, Kshitij Dwivedi, Dinesh Singh, Sanjeev Nara
similarity 0.50
Abstract

Convolutional neural networks (CNNs) and vision transformers are both used to model the human visual system, but whether the two architectures diverge at a specific point in network depth is unclear. We compared six CNNs and two vision transformers by computing the Pearson correlation (r) between each model's predicted and measured EEG response at every layer or block, in ten participants viewing 200 natural images. For the transformer models, we also tested four token representations, from the classification (CLS) token alone to CLS combined with all patch tokens. CNNs showed strongest correspondence at the earliest layers, weakening at deeper layers, particularly later in the post-stimulus response. Transformers instead sustained strong correspondence at their deepest blocks, though not at their earliest ones. This advantage depended on token representation: pooled representations gave weaker peak correlations (r approx 0.48-0.51) than representations retaining all patch tokens (r=0.640 for CLIP-ViT-B/32, r=0.656 for DINOv2-ViT-B/14). Controlled comparisons showed architecture, not training objective, drove this effect: MoCo-v1 and ResNet-50 (matched architecture) performed nearly identically (r=0.673, 0.670), whereas CLIP-RN50 and CLIP-ViT-B/32 (matched objective) diverged until patch tokens were preserved. We propose that CNN training's classification bottleneck compresses brain-relevant information at depth, unlike transformers' self-attention and non-classification objectives. A spatial topography analysis showed a common occipital-dominant pattern across all models, indicating these differences reflect signal strength and persistence rather than distinct brain regions. Patch-preserving transformer representations sustain brain-predictive correspondence where CNNs collapse.

0.16

Modeling Visual Cortex with Layerwise Multiscale Manifold Capacity Maximization

cs.CV q-bio.NC ·2026-11-01 ·Thomas Yerxa, SueYeon Chung, Eero P. Simoncelli
similarity 0.50
Abstract

Task-optimized deep neural networks have risen to prominence as the most predictive phenomenological models of responses in primate visual cortex, but leave much to be desired from the perspective of biological plausibility. One such limitation is the reliance on precise credit assignment through global backpropagation of error signals. Recent work has shown that this weakness can be circumvented by requiring each subsequent stage to solve a distinct and increasingly complex task, allowing for layerwise local learning signals. We propose a novel strategy for crafting such intermediate losses that uses an efficient coding framework formulated in terms of manifold capacity, and which can be computed using a sequence of {canonical cortical computations}. In particular, we leverage the relationship between the multiscale nature of visual signals and the dilation of receptive field sizes in cascaded visual representations to modulate complexity, allowing for the reapplication of these common loss computations at each stage of the hierarchy. We evaluate our approach on its ability to predict neural datasets spanning three areas of the ventral stream hierarchy in macaques, as well as human psychophysical data on an object classification task. We find that our unsupervised layerwise model matches or exceeds the performance of competitive architecture-matched baselines on all evaluations considered.

0.20

What Makes a Latent Space Shared? Direct and Recoverable Compatibility in Motion VAEs

cs.AI cs.CV ·2026-10-12 ·Iago Brito, Walcy Rios, Lucas Fernandes, Kaiki de Oliveira, Arlindo Galvão Filho
similarity 0.49
Abstract

A shared latent space is often assumed to make representations from different schemas directly interchangeable, yet sharing parameters, dimensionality, or a prior does not determine how corresponding instances are expressed in latent coordinates. We show that this assumption conflates recoverable correspondence, direct compatibility, and decoder reuse. In a controlled multi-schema motion VAE on HumanML3D, even identical inputs yield only $5.9$--$17.7\%$ raw Recall@1 and $0.221$--$0.484$ paired-token cosine, despite correspondence being almost perfectly recoverable after linear calibration. Cycle consistency strengthens recoverability but still leaves direct retrieval and token agreement near zero. In contrast, only $64$ paired clips raise raw Recall@1 to $88.4$--$90.3\%$ and token cosine to approximately $0.82$, making most post-hoc alignment unnecessary, although crossed decoding remains substantially worse than native reconstruction. We further find that reducing measured marginal discrepancy does not imply schema invariance. These results show that a shared backbone or prior does not ensure directly compatible latent coordinates, while limited paired correspondence can substantially improve direct compatibility in the tested setting.

0.26

How Long Is the Coast of a Neuron? Resolution-Dependent Interaction Order and Computational Capacity

cs.NE q-bio.NCcond-mat.dis-nncond-mat.stat-mech+1 ·2026-10-14 ·Nima Dehghani
unusual similarity 0.49
Abstract

Mandlebrot's coastline paradox reframes measurement as a joint property of an object and its ruler. Here we ask how observational resolution shapes the apparent structure of neural computation. We introduce a scale-indexed decomposition that separates the predictive information captured by direct dependencies from the residual outside that model family. Exact constructions show that projection can convert higher-order dependence into pairwise structure. Projection can increase the direct fraction of surviving information even as total predictive information decreases. A schematic branch-to-soma cascade grounds these effects in local coincidence detection, saturating integration and somatic mixing. Resolving organized nonlinear subunits improves approximation and raises effective representational rank. In a nested hierarchy, these gains occur without increasing information about the target. This distinction separates information availability from access by a restricted readout. A point-neuron or direct population model therefore establishes an effective description at a specified scale. It does not uniquely identify the finer computation that generated the observed relation. Which resolution is adequate depends on the observable and perturbation of interest.

0.42

ReGraph: A Computational Account of Emergent Generalization in the "what" and "where" Dual Visual Streams

cs.NE cs.AIq-bio.NC ·2026-10-06 ·Hyewon Kang, Jungmin Lee, Ilgyu Lee, Seok-Jun Hong
benchmark similarity 0.49
Abstract · 29 pages, 6 figures

Where generalization capacity--the ability to extract context-invariant relational structures--first emerges remains a central question in AI and neuroscience. The foundation for this capacity lies upstream of the hippocampus, within the entorhinal cortex, where parallel pathways dissociate relational structure in the medial entorhinal cortex (MEC) from sensory content in the lateral entorhinal cortex. However, as Eichenbaum argued, such factorization likely originates earlier, driven by the segregation of the dorsal ('where') and ventral ('what') visual streams. Supporting this, grid-like firing patterns--a signature of MEC (context-invariant codes)--also appear in preceding neocortical regions along the dorsal pathway. Yet, how such representations are computationally formed along upstream pathways remains unknown. To investigate this in silico, we developed ReGraph, a recurrent dual-stream graph model with biological inductive biases, including retina-driven stream-specialized encoding, dorsal-to-ventral modulation, and dynamic lateral connectivity. Trained on the action benchmark Something-Something V2, ReGraph revealed a pathway-specific emergence of relational mapping: context-invariant codes and grid-like spatial bases uniquely co-emerged along the extended dorsal stream. In contrast, their absence in single-stream, unmodulated variants, and standard baselines implies that these inductive biases are prerequisites for relational structures. Crucially, our post-hoc analyses demonstrated that these grid-like bases serve as reusable routing templates for information processing via lateral connectivity. Together, our findings provide a computational account that generalization may not be a faculty that emerges abruptly within a dedicated region, but a property that already takes shape as sensory information is parsed into factorized streams of hierarchical visual processing.

0.32