CS Paper Radar

The Computational Value of Sensory-Aligned Receptive Fields Depends on Neuronal Expressivity

Agnese Adorante, Aaron Spieler, Anna Levina

cs.NE cs.LGq-bio.NC ·submitted 2026-09-22 18:27 UTC · v1

Abstract

Biological sensory neurons have selective receptive fields organized along meaningful stimulus coordinates, such as frequency, motion direction, or retinotopic position. Such structure may arise from efficient coding and biological constraints on activity, connectivity, and wiring, as computational studies of simple neurons have shown across modalities. This raises a question: do structured receptive fields confer a computational advantage beyond resource efficiency itself, and does this advantage persist when individual neurons are highly expressive? We address this question in recurrent networks of Expressive Leaky Memory neurons, where we can independently vary neuronal complexity and the organization of feed-forward receptive fields. Across auditory and event-based visual classification tasks, receptive fields aligned with a task-relevant sensory coordinate improve test accuracy relative to budget-matched random receptive fields. This advantage disappears when sensory coordinates are scrambled, or when receptive fields follow task-irrelevant coordinates, showing that the benefit comes from alignment with task geometry rather than restricted connectivity alone. Increasing neuronal complexity reduces the performance advantage of structured receptive fields. Finally, generic synaptic sparsity regularization induces input selectivity and partially recovers performance, but remains substantially below explicitly structured receptive fields, suggesting that sparsity alone is insufficient to recover the full computational benefit of task-aligned receptive fields. Together, our results show that appropriate receptive fields can serve as a computational prior beyond sparsity itself, and that their value depends on the computational expressivity of individual neurons.

Closest papers

Nearest neighbours by meaning across the whole index.

Modeling Visual Cortex with Layerwise Multiscale Manifold Capacity Maximization

cs.CV q-bio.NC ·2026-11-01 ·Thomas Yerxa, SueYeon Chung, Eero P. Simoncelli
similarity 0.61
Abstract

Task-optimized deep neural networks have risen to prominence as the most predictive phenomenological models of responses in primate visual cortex, but leave much to be desired from the perspective of biological plausibility. One such limitation is the reliance on precise credit assignment through global backpropagation of error signals. Recent work has shown that this weakness can be circumvented by requiring each subsequent stage to solve a distinct and increasingly complex task, allowing for layerwise local learning signals. We propose a novel strategy for crafting such intermediate losses that uses an efficient coding framework formulated in terms of manifold capacity, and which can be computed using a sequence of {canonical cortical computations}. In particular, we leverage the relationship between the multiscale nature of visual signals and the dilation of receptive field sizes in cascaded visual representations to modulate complexity, allowing for the reapplication of these common loss computations at each stage of the hierarchy. We evaluate our approach on its ability to predict neural datasets spanning three areas of the ventral stream hierarchy in macaques, as well as human psychophysical data on an object classification task. We find that our unsupervised layerwise model matches or exceeds the performance of competitive architecture-matched baselines on all evaluations considered.

0.20

Early Learning Shapes Later Directions Of Representation Change In Continual Learning

cs.LG ·2026-09-28 ·Yuantao Deng, Jinnuo Liu, Kaizhen Tan, Yuchen Liu
code/datahot direction similarity 0.61
Abstract · 35 pages, 7 figures

Representations continually change as a network learns new tasks. We ask whether early representational changes naturally form a geometric structure that continues to shape later learning. We identify a low-dimensional subspace of early representation drift, which we call a scaffold, and test whether it is reused across subsequent tasks. Across four pretrained visual encoders and two datasets, later representational changes consistently favor this early-defined subspace over matched random alternatives. This reuse is history-dependent: when networks experience different early tasks but identical later training inputs, each network preferentially reuses the scaffold induced by its own learning history. The same preference appears in individual optimizer updates, even though the network's dominant local response directions shift away from the original scaffold. Finally, constraining motion within the scaffold slows new-task acquisition more than matched random constraints, while effects on old-task retention are less consistent. In summary, these results suggest that early experience leaves a persistent geometric imprint on how neural networks adapt to future tasks.Code is available at https://github.com/YuantaoDeng/latent-scaffold.

0.53

ReGraph: A Computational Account of Emergent Generalization in the "what" and "where" Dual Visual Streams

cs.NE cs.AIq-bio.NC ·2026-10-06 ·Hyewon Kang, Jungmin Lee, Ilgyu Lee, Seok-Jun Hong
benchmark similarity 0.60
Abstract · 29 pages, 6 figures

Where generalization capacity--the ability to extract context-invariant relational structures--first emerges remains a central question in AI and neuroscience. The foundation for this capacity lies upstream of the hippocampus, within the entorhinal cortex, where parallel pathways dissociate relational structure in the medial entorhinal cortex (MEC) from sensory content in the lateral entorhinal cortex. However, as Eichenbaum argued, such factorization likely originates earlier, driven by the segregation of the dorsal ('where') and ventral ('what') visual streams. Supporting this, grid-like firing patterns--a signature of MEC (context-invariant codes)--also appear in preceding neocortical regions along the dorsal pathway. Yet, how such representations are computationally formed along upstream pathways remains unknown. To investigate this in silico, we developed ReGraph, a recurrent dual-stream graph model with biological inductive biases, including retina-driven stream-specialized encoding, dorsal-to-ventral modulation, and dynamic lateral connectivity. Trained on the action benchmark Something-Something V2, ReGraph revealed a pathway-specific emergence of relational mapping: context-invariant codes and grid-like spatial bases uniquely co-emerged along the extended dorsal stream. In contrast, their absence in single-stream, unmodulated variants, and standard baselines implies that these inductive biases are prerequisites for relational structures. Crucially, our post-hoc analyses demonstrated that these grid-like bases serve as reusable routing templates for information processing via lateral connectivity. Together, our findings provide a computational account that generalization may not be a faculty that emerges abruptly within a dedicated region, but a property that already takes shape as sensory information is parsed into factorized streams of hierarchical visual processing.

0.32

Pixels over Symbols: Sensory Realism Improves Behavioral Alignment in Models of Cognition

cs.AI q-bio.NC ·2026-11-07 ·Hao Yuan Bai, Xiaoxuan Lei, Zihan Weng, Motahareh Pourrahimi, Lucas Gomez +1 more
similarity 0.59
Abstract

Computational models of cognition have traditionally relied on abstract symbolic inputs that approximate sensory experiences with simplified vector representations. This simplification reflects historical computational constraints and a long-held assumption that the details of sensory realism are incidental to the study of cognitive processes. Recent advances have enabled the development of sensory-realistic models capable of performing diverse cognitive tasks, yet whether sensory realism meaningfully shapes cognition has not been formally established. We address this gap through a large-scale empirical comparison of abstract and sensory-realistic neural network models trained on a battery of memory-dependent decision-making tasks. Our results show that sensory-realistic models mirror human behavioral response patterns significantly more than their abstract counterparts, though a substantial gap to human internal behavioral consistency remains even for the best tested models. By analyzing the internal dynamics of these systems, we find that the learned recurrent weights are not the main source of this divergence. In fact, frozen abstract-trained weights driven by sensory-realistic inputs recover the human consistency of sensory-realistic models. The resulting dynamics are also within the seed-to-seed variability of sensory-realistic models. Together, our findings argue that across a broad set of visual working-memory tasks, sensory realism accounts for a substantial portion of the model-human behavioral agreement, more than the choice of sequence architecture and comparable to the choice of vision encoder. These results suggest that findings obtained from abstract task models in neuroscience should be more carefully interpreted in light of their potential behavioral limitations.

0.17

Parallel Time-Aligned Spiking Self-Attention for Consistent Integer-Valued Training and Spike-Driven Inference

cs.NE ·2026-10-02 ·Peng Xue, Wei Fang, Kaiwei Che, Qingyan Meng, Zhengyu Ma +2 more
similarity 0.59
Abstract

Integer-valued leaky integrate-and-fire (I-LIF) neurons and spike firing approximation (SFA) reduce temporal training cost by representing spike trains as firing counts and normalized firing rates, respectively. However, applying spiking self-attention (SSA) directly to these compressed query, key, and value representations introduces cross-time interactions that are absent during spike-driven inference. We term this operator-level discrepancy Temporal Interaction Mismatch (TIM). We propose Parallel Time-Aligned Spiking Self-Attention (PT-SSA), which reconstructs consecutive virtual spike slices from either I-LIF counts or SFA firing rates, computes attention only between time-aligned slices in parallel, and sums the per-step outputs. To accommodate the reduced attention output scale under SFA, we further introduce Adaptive PT-SSA, which learns a positive per-block rescaling before the output SFA neuron to improve firing-level utilization. Experiments on CIFAR-10, CIFAR-100, and ImageNet-1K show that the proposed methods substantially reduce train--inference mismatch. On CIFAR-100 with I-LIF, PT-SSA reduces the mean Top-1 gap from 1.85 to 0.29 percentage points. On ImageNet-1K, Adaptive PT-SSA reduces the Top-1 gap from 27.78 to 0.06 percentage points and achieves 74.53\% spike-driven Top-1 accuracy. A Triton-fused PT-SSA training kernel retains a $2.91\times$ throughput advantage over recurrent LIF SSA.

0.19

Two Routes to the Middle: Placement Search and Brain Readouts Converge on Where Continual Learners Should Specialize

cs.LG cs.CV ·2026-10-01 ·Yuan Huang, Zihan Chen, Runbin Zhang, Hongwei Ding, Changzeng Fu +1 more
similarity 0.59
Abstract · 21 pages, 12 figures

Continual learners that keep a task-specific adapter in every block of a pre-trained vision transformer accumulate storage linearly with the number of tasks; keeping task-specific adapters in only a few blocks curbs this growth but raises the question of where to place them. We investigate this question from two perspectives. Algorithmically, training all contiguous four-block placements yields an inverted U: final accuracy peaks at intermediate depth and varies by up to 3.5 percentage points (pp), while inexpensive criteria based on weight spectra or activation statistics favor the deepest blocks. From neuroscience, the hierarchical organization and intermediate-stage plasticity of the visual cortex motivate us to ask whether a measurement taken outside the learner can guide layer specialization without placement search. LS-B observes the first tasks through a frozen fMRI encoding model of twelve human visual areas and commits task-specific capacity once to the blocks whose readouts vary most across tasks relative to their stable structure. Across three ViT-B/16 backbones, LS-B yields stable, backbone-specific allocations. On the two backbones with placement search, AugReg and iBOT, the selected blocks overlap the intermediate-depth region identified by search. Under matched storage and observation budgets, the selected blocks outperform the shallowest and deepest four-block configurations. On Split ImageNet-R, LS-B uses 60% of full-BiLoRA adapter storage while remaining within 1.5 pp of its final accuracy. The allocation requires no labels or backpropagation, adds under 0.6% runtime, and exhibits backbone-specific cortical signatures.

0.23

Learning infinite context windows in recurrent architectures via spatial neural computing

cs.LG eess.SYq-bio.NC ·2026-10-07 ·Aleix Salvador-Pomarol, Arthur N. Montanari, Earl K. Miller, Adilson E. Motter, Jorge Cortés
similarity 0.58
Abstract · 12 pages, 4 figures

Recurrent neural networks (RNNs) offer linear-time scaling with sequence length while requiring only constant memory, yet they struggle to capture long-range dependencies due to vanishing gradients and limited receptive fields. To address these limitations, we introduce a second-order recurrent model in which the standard neuron-to-neuron communication is replaced by a spatially evolving field governed by (discretized) partial differential equations. Drawing inspiration from the role of cortical waves in brain computation, this mechanism allows structured spatiotemporal patterns to serve as an implicit, high-capacity memory. We show that the resulting model is equivalent to a structured infinite-order RNN in which the current state depends explicitly on its entire history of past states, yielding an effectively unbounded receptive field with a fixed number of parameters. We further derive constructive conditions to ensure marginal stability, constraining the gradient spectrum on the unit circle and thereby eliminating vanishing and exploding gradients. Empirically, the proposed architecture outperforms other recurrent models on long-horizon benchmarks while using substantially fewer parameters, demonstrating that spatial dynamics can effectively bridge the gap between efficient inference and long-term memory.

0.31

FlowGenesis: Representation-Driven Self-Organization of Neural Architectures

cs.LG ·2026-10-26 ·Maksim A Kazanskii
similarity 0.58
Abstract

Deep neural networks typically rely on architectures specified before training, despite the continuous evolution of their internal representations. We introduce FlowGenesis, a framework in which neural architectures adapt through neuron birth and pruning guided by Representation Flux, a local statistic capturing changes in minibatch mean activations. Neurons exhibiting persistently high Flux undergo splitting designed to approximately preserve the network function, while neurons with low Flux and low activity are pruned. This mechanism allows network capacity to evolve without prescribing a target architecture or performing a separate architecture search. Experiments on image and text classi- fication show that networks initialized with different sizes approach similar final neuron-count ranges within each dataset while maintaining comparable predictive performance. We further extend FlowGenesis to continual learning, where representation-guided structural adaptation reduces the neurons and parameters allocated to successive tasks while maintaining accuracy comparable to fixed-width Progressive Neural Networks in a task-incremental learning setting. These findings demonstrate that neural architectures can adapt through feedback between representation learning and structural change, suggesting an alternative to prescribing network capacity before training.

0.23