CS Paper Radar

Evaluation Resolution Confounds Learning-Rule Comparisons in Model-Brain RSA of Early Visual Cortex

Nils Leutenegger

cs.LG q-bio.NC ·submitted 2026-08-11 15:50 UTC · v3

Abstract

Studies of biologically plausible learning rules (feedback alignment, predictive coding, STDP) train small networks on $32\times32$ images and compare them with brain responses to naturalistic stimuli presented at much higher resolution. We find that a common result in this setting, that untrained or locally trained networks rival or beat backpropagation at early visual cortex, depends on the resolution at which networks are evaluated. In representational similarity analysis (RSA) against human V1, the gap between an untrained and a backprop-trained network grows monotonically from $-0.001$ (95% CI $[-0.011, 0.009]$) at the 32 px training resolution to $+0.030$ $[0.018, 0.042]$ at 224 px (six resolutions, 5 seeds; per-subject RSA on cross-run stimulus pairs). The growth reproduces for three further learning rules, directionally in single-seed macaque electrophysiology, along training, and for an ImageNet ResNet-50 and a Swin-Tiny transformer, which also align best at low resolution although trained at 224 px. Gabor and pixel structure, the normalization state of the untrained baseline, and a global brightness statistic do not account for it. Against a criterion fixed before the analysis, limiting image content to 32 px while the input still grows does not shrink the V1 gap ($+0.035$ $[0.023, 0.046]$ vs. $+0.030$ at 224 px; ratio 1.14 $[1.03, 1.34]$): it arises once the input exceeds the training resolution, through a decline of backprop. A single luminance value per image reaches $ρ= 0.054$ against V1, matching the untrained network (0.053), which bounds what RSA on globally pooled features can resolve at V1 in this dataset. The one learning effect that holds across resolution is backprop above untrained at LOC. Comparisons of learning rules or architectures at early visual cortex need to control, and report, the evaluation resolution.

Comment
v3: RSA per subject on cross-run stimulus pairs; V1 gap +0.044 -> +0.030 [0.018, 0.042], still ~0 at 32 px. Leave-one-subject-out lower bound replaces the withdrawn noise ceiling. Sec. 3.5 reversed: with content limited to 32 px the V1 gap is as large (R = 1.14), so it follows input size, not detail above 32 px. Figures recomputed. Main result holds

Closest papers

Nearest neighbours by meaning across the whole index.

Analysis choices decide whether untrained networks beat trained ones at V1: a multiverse analysis of model-brain RSA

cs.LG q-bio.NC ·2026-11-11 ·Nils Leutenegger
code/data similarity 0.75
Abstract · 31 pages, 6 figures; code and data pipeline: https://github.com/nilsleut/ridge-readout

Untrained convolutional networks have been reported to rival trained networks as models of visual cortex, and we had reported an untrained-network advantage over backpropagation at V1 in THINGS fMRI, together with a dependence on evaluation resolution. Under a pre-specified ridge readout, the untrained network scored below the trained one at every pooling grid (median r difference -0.043 at g = 4, 0 of 5 seeds positive), and also in MSE skill and per-fold r. In exploratory analyses, across 36 combinations of unit normalisation, distance measure and voxel normalisation (28 distinct analyses), the sign of the V1 contrast depended on the analysis (intervals above 0 in 5 and below 0 in 9 distinct analyses), and in our networks the resolution dependence of the original convention appeared in only one cell. Raw correlation-distance representational similarity analysis (RSA) covaried with the alignment of model and voxel patterns with the unit-mean profile (Spearman 0.97 across our networks, 0.79 across six ImageNet architectures); the advantage was absent from the component orthogonal to the mean profile, and the parallel component alone did not resolve it. On the unweighted angular component of centred patterns, trained networks were ahead at V1 (our network and 5 of 6 ImageNet models) and at LOC (all seven); the norm-weighted angular term was not resolved. A luminance-and-contrast bank scored above every network under ridge (point estimates), and adding network features under a shared penalty did not improve prediction. Intervals are stimulus-bootstrap intervals for three subjects, and 6 pre-stated predictions were not met. We recommend reporting sensitivity to centring over the evaluated stimuli on both sides, reporting components separately, and including a small multiverse and simple baselines.

0.36

Modeling Visual Cortex with Layerwise Multiscale Manifold Capacity Maximization

cs.CV q-bio.NC ·2026-11-01 ·Thomas Yerxa, SueYeon Chung, Eero P. Simoncelli
similarity 0.59
Abstract

Task-optimized deep neural networks have risen to prominence as the most predictive phenomenological models of responses in primate visual cortex, but leave much to be desired from the perspective of biological plausibility. One such limitation is the reliance on precise credit assignment through global backpropagation of error signals. Recent work has shown that this weakness can be circumvented by requiring each subsequent stage to solve a distinct and increasingly complex task, allowing for layerwise local learning signals. We propose a novel strategy for crafting such intermediate losses that uses an efficient coding framework formulated in terms of manifold capacity, and which can be computed using a sequence of {canonical cortical computations}. In particular, we leverage the relationship between the multiscale nature of visual signals and the dilation of receptive field sizes in cascaded visual representations to modulate complexity, allowing for the reapplication of these common loss computations at each stage of the hierarchy. We evaluate our approach on its ability to predict neural datasets spanning three areas of the ventral stream hierarchy in macaques, as well as human psychophysical data on an object classification task. We find that our unsupervised layerwise model matches or exceeds the performance of competitive architecture-matched baselines on all evaluations considered.

0.20

Layerwise Learning-Rate Scaling in Vision Networks: Effective-Rate Identification, Power-Mean Aggregation, and Budget Dependence

cs.LG ·2026-11-16 ·Chandini Vysyaraju, Raghuvir Duvvuri, Radu Timofte, Dmitry Ignatov
similarity 0.59
Abstract · 10 pages, 6 figures. Supplementary material (15 pages) included as an ancillary file. Data and analysis scripts: https://doi.org/10.5281/zen…

Layerwise learning-rate scaling assigns different learning rates to different parts of a network. Although established in fine-tuning pretrained models, its effects in vision networks trained from scratch are commonly assessed by training each candidate recipe. We show that, in this setting, the rate-related component of a layerwise strategy is well summarized by an effective learning rate computable directly from its multiplier profile. Across matched-control comparisons, changes in test accuracy closely track changes in training loss (r = -0.996), which supports an optimization-progress explanation of the short-budget rate effect. Increasing the untuned base learning rate tenfold shifts the rate-matched contrast upward in all six tested architectures (p = 0.016) and increases the number of architectures in which a published discriminative schedule, excluded from model fitting, outperforms the uniform control from two to five of six (p = 0.031). Within the evaluated strategy space, a parameter-weighted power mean with exponent 0.35 (95% confidence interval [0.20, 0.50]) accounts for 42.6% of within-architecture variance, compared with 36.4% for the arithmetic mean. Ranking candidates by this scalar without training improves the selected candidate's mean percentile rank from the 26th to the 10th and reduces its mean accuracy shortfall relative to the best candidate from 3.70 to 1.12 points. The rate-related effect decays as the training budget increases; relative to tuned baselines, the overall strategy effect falls within the prespecified practical-equivalence margin in two of three architectures. A separate grouping-related contrast can persist at long budgets, although its direction and magnitude depend on the architecture.

0.43

Neuron-Level Architecture Growth: A Controlled Evaluation for EEG Time-Series Decoding

eess.SP cs.HCcs.LGcs.NE ·2026-09-27 ·Adam Mounir, Stella Douka, Arnault H. Caillet, Bruno Aristimunha, Sylvain Chevallier
similarity 0.59
Abstract · 5 pages, 3 figures, 2 tables. Submitted to ICASSP 2027

Convolutional EEG decoders are trained at a fixed width, usually set by their authors on other data. Growing methods add neurons during training where the loss could decrease the most, but whether they improve compared to a reference width is untested on EEG. Here, we grow three convolutional backbones on 12 motor-imagery datasets under three protocols and compare each with its reference model per subject. The growing ShallowFBCSPNet scores 2.9 points above its reference model with only half the parameters (0.57x), SCCNet changes by at most 1.2 points. Deep4Net growing models show decreased accuracy, but they require adaptation that prevent to compare faithfully the results. These differences follow the selection step, which keeps a candidate neuron relying on a dynamic threshold from singular values decomposition. Overall, these results suggest that growth helps when its criterion can rank the candidate neurons, and that the rate of skipped neuron addition tells where a decoder can be grown small from scratch.

0.35

Brain-Aligned Latent Actions: Imposing Premotor Cortex Geometry on a Robot's Action Vocabulary

cs.RO cs.LG ·2026-11-08 ·Amit Jadhav
similarity 0.58
Abstract · 16 pages, 3 figures, 6 tables. Robot manipulation results (CALVIN) will be added in a later version

Robot policies increasingly act through a small vocabulary of discrete latent actions learned from unlabeled video, yet training leaves open which motion receives which code. We find this choice highly unstable: two independently trained runs of Moto's latent motion tokenizer assign the same robot clips to nearly unrelated codes (adjusted Rand index 0.001), and evaluating one tokenizer in bf16 instead of fp32 reassigns 74% of its tokens. We use this freedom to impose the representational geometry of macaque ventral premotor cortex (PMv), recorded while a monkey drew shapes, on the tokenizer's code assignment, with a straight-through similarity loss trained alongside its reconstruction objective on robot video. In one training run per target, on five shape categories the loss never saw, the PMv-trained tokenizer reaches a representational similarity of 0.52 with PMv, against 0.20-0.26 for released and no-prior tokenizers and 0.42 for the 95th percentile of random kinematic encoders. Kinematic, language, and random targets do not transfer (-0.17 to -0.03). Two results limit the claim: a hand-picked kinematic descriptor, the stroke's initial direction, matches PMv better (0.65), and a label-shuffled PMv control scores 0.41. We specify the downstream manipulation test, on a CALVIN pipeline that reproduces Moto's published results, before running it.

0.30

Alignment of visual brain encoding models with individual perceptual similarity judgments

cs.CV q-bio.NC ·2026-10-15 ·Daniel Chong, Haomiao Chen, Keith W Jamison, Raja Marjieh, Ilia Sucholutsky +2 more
similarity 0.57
Abstract

Encoding models that predict brain activation responses to images have traditionally been trained to maximize accuracy when predicting observed brain data. However, their internal representations may not reflect individual human perception. In this work, we investigate whether we can fine-tune brain encoding models to better align their representations with individuals' similarity judgments. Using similarity judgments of 100 images from 8 participants in the Natural Scenes Dataset (NSD), we adapt a generic human similarity judgment model (DreamSim) separately for each participant to obtain personalized perceptual similarity models. These models are then used to obtain subject-specific similarity structure across images in their encoding model training sets. We fine-tune each participant's baseline encoder in two conditions, aligning its representations with either generic or personalized similarity judgments, with the same initialization and objective. We find that baseline brain encoders reflect within-subject perceptual similarity more strongly than measured fMRI responses and preserve intersubject correlations present in measured fMRI, but fail to reflect those of human perceptual similarity judgments. Fine-tuning with generic, population-level perceptual similarity information results in better alignment with human judgments than baseline encoders, but these models still fail to capture intersubject correlations of human judgments. However, fine-tuning with personalized perceptual similarity information results in models that are the most aligned with human perceptual similarity judgments and also reflect intersubject correlations of human judgments. Our framework provides an approach for bootstrapping human similarity judgment data into more individualized, human-aligned brain encoding models of vision.

0.19

Breaking the Central Bias: Spatially Partitioned Experts for Coordinate-Based Neuroevolution

cs.NE cs.CVcs.LG ·2026-09-10 ·Romain Claret, Arthur Gygax, Michael O'Neill, Paul Cotofrei, Michael Palma Mendes +1 more
publishedbenchmark similarity 0.56
Abstract · 15 pages, 4 figures, 1 table. Author's accepted manuscript, accepted at the BIOMAP workshop (BIO-inspired Methods for Pattern Recognition) o…

Evolvable-Substrate HyperNEAT (ES-HyperNEAT), a bio-inspired indirect encoding that determines neuron placement and connection weights from spatial coordinates, exhibits a failure mode on MNIST as a diagnostic benchmark. Because input pixels map to a coordinate space centered at the origin, evolved networks converge on a small central cluster of input pixels, a spatial-concentration bias; prior work observed only 21% mean accuracy in this regime. Is this bias an optimization artifact or an architectural ceiling? Inspired by Mixture-of-Experts (MoE) principles, we partition the input into non-overlapping spatial segments, each assigned to a separately evolved specialist network. With 13 such experts, this design reaches 43% mean accuracy, a 106% relative improvement over the baseline. The architectural gain does not depend on data-driven aggregation: equal-weighted averaging, which uses no validation data, already yields a 70% improvement; the gain comes from partitioning, not the weighting. Receptive-field analysis shows the mechanism: partitioning forces evolution to discover features across the entire image, expanding active pixel coverage from 4% to 79%. Absolute accuracy stays below gradient-trained baselines, but the relative gain points to central bias, not the evolutionary search. Two tools are designed to generalize beyond MNIST: a receptive-field diagnostic for silent input-coverage collapse, and a spatial-partitioning remedy that restores coverage.

0.66

Brain-to-Image Generation: Reconstructing Visual Stimuli from EEG using Generative Adversarial Networks

cs.CV cs.AI ·2026-09-13 ·Harshit Goyal
similarity 0.56
Abstract

Reconstructing visual stimuli from electroencephalography (EEG) is difficult because scalp measurements have high temporal but limited spatial resolution, and paired EEG-image datasets remain small relative to modern generative-model training corpora. We present a reproducible single-subject baseline on THINGS-EEG2 that first tests the more defensible question of whether EEG can retrieve the viewed stimulus in a visual embedding space. A compact temporal-spatial convolutional encoder maps repetition-averaged EEG (63 by 250) to provided 512-dimensional ViT-B/32 image features. Model selection uses a concept-disjoint validation split, and final evaluation uses the official 200-image, 200-concept test gallery. Across three training seeds, the model obtains 12.83 +/- 0.58%, 39.17 +/- 1.76%, and 58.00 +/- 1.73% image recall at 1, 5, and 10 (mean +/- sample standard deviation), compared with analytical chance levels of 0.5%, 2.5%, and 5.0%. A session-balanced ablation shows that averaging more test repetitions generally improves ranking. Applying the Subject 01 model to the other nine subjects without adaptation causes a sharp performance drop, exposing subject specificity. We further report exploratory stress tests of direct conditional generators trained without external visual weights: single-subject and ten-subject variants produce noise-dominated outputs, with early validation improvements reversing after one to four epochs. Finally, we distinguish direct reconstruction from semantic rendering with a pretrained diffusion prior. The results support above-chance coarse semantic decoding under a closed-set, repetition-averaged protocol, but do not support faithful recovery of stimulus pixels.

0.27