CS Paper Radar

NeuroCBIR: A Fast and Accurate Image Retrieval System for Whole-Brain and Region-Specific MRI

Felix Nieto-del-Amor, Jingru Fu, J. -Sebastian Muehlboeck, Eric Westman, Daniel Ferreira, Rodrigo Moreno

cs.CV cs.LGq-bio.NC ·submitted 2026-10-05 15:22 UTC · v1
code/data

Abstract

Content-based image retrieval (CBIR) in neuroimaging enables the identification of structurally similar brain scans, supporting diagnosis, prognosis, and treatment planning; however, existing methods are often limited to small datasets, single brain regions, or coarse class labels, thereby restricting their clinical utility and generalizability. Here, we present NeuroCBIR, a framework for fast and flexible retrieval of both whole-brain and region-specific 3D T1w MRI scans. A total of 103 cortical and subcortical regions are extracted to enable both whole-brain and region-level queries. NeuroCBIR leverages latent representations learned by a variational autoencoder (VAE) combined with contrastive learning, producing scan-specific embeddings that capture anatomical patterns. These embeddings were evaluated for subject re-identification, zero-shot age prediction, and zero-shot multi-class pathology stratification. Re-identification performance was high across both whole-brain and brain-region levels (mean average precision across the top-5 retrieved images (mAP@5) >= 98.4%), with robust generalization across datasets and acquisition conditions. While NeuroCBIR is not trained for age prediction or pathology stratification, zero-shot evaluations for these two tasks demonstrate that the embeddings encode meaningful information for downstream tasks. Embedding extraction on a 4-core CPU required approximately 18.7 s per scan, whereas similarity search was effectively instantaneous (less than 0.01 s). NeuroCBIR is publicly available for brain MRI with more than 26,000 precomputed T1w MRI embeddings. It supports reproducible research, region-specific flexibility, and clinically meaningful personalized diagnostic support. The software is available at https://github.com/minnelab/NeuroCBIR.

Comment
Neuroimaging, Content-Based Image Retrieval, MRI, Zero-Shot Learning

Closest papers

Nearest neighbours by meaning across the whole index.

FSS-UBrain: Multi-region Few-Shot Brain Tumor MRI Segmentation

cs.CV ·2026-09-26 ·Truong Viet Vu, Nguyen Phuc Nguyen, Dang Thi Thu Hang, Tran Thien Thanh, Vo Nguyen Quoc Bao +2 more
published similarity 0.70
Abstract · This work has been submitted to the Elsevier for possible publication. Copyright may be transferred without notice, after which this version…

Accurate delineation of whole tumor (WT), tumor core (TC), and enhancing tumor (ET) from multimodal magnetic resonance imaging remains challenging under limited annotation, cross-cohort variation, and severe target sparsity. We propose FSS-UBrain, a region-wise one-shot segmentation framework that uses a labeled positive support slice to condition binary query segmentation separately for WT, TC, and ET. Support-derived foreground and background descriptors guide query-feature adaptation, bottleneck interaction, decoder-side reconstruction, and boundary refinement. Episodic training additionally incorporates hard-negative and fully negative queries with empty-query regularization to suppress spurious foreground activation when the selected region is absent. Although inference operates on two-dimensional support--query slice pairs, checkpoint selection, threshold calibration, and final evaluation are performed after volumetric reconstruction. FSS-UBrain is evaluated on a held-out BraTS 2020 split and under target-supported cross-cohort protocols on BraTS 2023 and BraTS-Africa. Cases used as target support are excluded from the query cohorts, and no target-domain fine-tuning or test-time parameter updates are performed. On BraTS 2020, FSS-UBrain achieves volumetric Dice scores of 89.82%, 82.14%, and 77.42% for WT, TC, and ET, respectively, with corresponding 95th-percentile Hausdorff distance (HD95) values of 11.12, 9.01, and 4.46 mm. It also achieves the highest mean Dice and lowest finite-pair mean HD95 point estimates on BraTS 2023 and BraTS-Africa among the compared few-shot methods. These findings support target-conditioned few-shot segmentation while highlighting sensitivity to support selection and cohort-specific variation.

0.24

Brain2Vec: A Self-Supervised 3D Foundation Model for Structural Brain MRI

cs.CV cs.AIcs.LG ·2026-10-20 ·Spiros Maggioros, Guray Erus, Gareth Harman, George Aidinis, Alexander Getka +2 more
code/data similarity 0.66
Abstract

Deep learning models have become essential tools in medical imaging, yet training them end-to-end for diverse tasks remains challenging due to limited labeled data and computational constraints. To address this, we introduce Brain2Vec, a foundation model pre-trained on 74,425 T1-weighted Magnetic Resonance Imaging (MRI) scans pooled from multiple studies. Brain2Vec can serve both as a weight initialization backbone for fine-tuning downstream tasks and as a standalone feature extractor. We evaluated Brain2Vec in three tasks: downstream predictive performance, convergence speed, and embedding quality. Using Brain2Vec's pretrained weights as initialization and fine-tuning on labeled data for a brain age prediction task, results demonstrate improved final accuracy and faster convergence compared to random initialization. Additionally, its frozen embeddings outperform anatomically defined volumetric features on four of five downstream classification tasks and perform competitively with a state-of-the-art foundation model pre-trained on substantially more data. Lastly, we show that Brain2Vec encodes information primarily from a spatially localized and neuroanatomically meaningful set of brain regions, and that manipulating a subset of latent features produces monotonic, region-specific effects on the reconstructed anatomy. Brain2Vec code and trained models are available as an open-source package on https://github.com/CBICA/Brain2Vec. Users can apply Brain2Vec locally by installing the package, or directly through the NiChart cloud platform https://cloud.neuroimagingchart.com, where Brain2Vec features can be derived without requiring local installation or specialized infrastructure.

0.23

Beyond Representation Learning: A Systematic Study of Joint-Embedding Predictive Generation for 3D Brain MRI

cs.CV ·2026-08-28 ·Meng Zhou, Wenhao You, Yuxing Chen, Yueying Tian
similarity 0.66
Abstract · Preprint, code will be released after review

Joint-embedding predictive architectures (JEPAs) have primarily been developed for self-supervised representation learning. Denoising JEPA (D-JEPA) recently demonstrated strong generative capabilities on natural images, yet the applicability to 3D medical imaging remains unexplored. Building on the D-JEPA framework, we present Med-D-JEPA, a systematic adaptation and evaluation of joint-embedding predictive generation for 3D brain MRI. Med-D-JEPA operates on continuous latent tokens produced by a 3D KL-regularized adversarial variational autoencoder, and combines masked context prediction, representation-level alignment, per-token diffusion, and iterative next-set-of-token sampling. We evaluate unconditional and class-conditional generation quality on BraTS2019 and OASIS-1 datasets; downstream classification utility; and preliminary whole-tumor segmentation on BraTS2020. Across different generation settings, Med-D-JEPA achieves superior or competitive performance compared to several strong baselines on fidelity and diversity metrics. Compared to training with real samples, Med-D-JEPA-based synthetic pretraining improves classification AUC from 0.63 to 0.85 on BraTS2019 and from 0.78 to 0.87 on OASIS-1. In the segmentation study, pretraining on Med-D-JEPA samples improves Dice from 0.74 to 0.80 and reduces HD95 from 13.40 to 9.56 mm. These findings establish joint-embedding predictive generation as a promising direction for 3D medical image synthesis and encourage further research in this direction.

0.47

Systematic Morphometric Brain Autoencoding for Representation Learning in Neurodegenerative Diseases

cs.CV cs.LG ·2026-10-27 ·Nuria Balbás-Esteban, Albert Belenguer-Llorens, Elaheh Moradi, Jussi Tohka, Vanessa Gómez-Verdejo +1 more
similarity 0.65
Abstract

Early identification of neurodegenerative disorders using structural T1-weighted MRI is essential for timely clinical intervention, yet supervised deep learning models often struggle to generalize due to limited annotations and scanner-induced distribution shift. Self-supervised learning offers a promising alternative; however, existing approaches are largely adapted from natural image paradigms, relying on contrastive objectives or generic masked reconstruction with voxel-wise losses, which may not fully capture the distributed morphometric patterns associated with neurodegeneration. We introduce SYMBA (SYstematic Morphometric Brain Autoencoding), a self-supervised pretraining framework specifically designed for structural brain MRI. SYMBA integrates anatomically constrained masking, a multi-scale structural similarity (MS-SSIM) reconstruction objective, and a progressive curriculum that transitions from local anatomical detail to global brain morphology. These components explicitly incorporate neuroanatomical structure into the self-supervised objective rather than relying on generic image priors. Pretrained on 6,953 T1-weighted MRI scans from nine heterogeneous cohorts, SYMBA is evaluated using frozen linear probing on two independent unseen datasets, enabling a strict assessment of out-of-distribution generalization. Across multiple clinical tasks, SYMBA consistently outperforms recent neuroimaging foundation models, achieving 0.930 AUROC for Alzheimer's disease classification, 0.772 for mild cognitive impairment conversion prediction, and strong performance in amyloid status prediction. In low-label regimes, SYMBA demonstrates high sample efficiency, reaching approximately 0.90 AUROC for AD/CN classification using only six labeled subjects.

0.18

Dual-VaDE: Efficient and Robust Prototype Learning via Variational Dual-Domain Mixture Priors

cs.LG ·2026-11-03 ·Wugeng Zheng, Song Wang, Jian Wang
theory similarity 0.64
Abstract

In this paper, we propose Dual-VaDE, an unsupervised generative framework for learning interpretable anatomical prototypes from unlabeled medical images. In contrast to previous variational methods that rely on a single spatial latent code, which often entangles global geometry with local textures and fails to capture their cross-domain dependencies, Dual-VaDE introduces a dual-domain Gaussian mixture prior that explicitly couples spatial anatomy with phase-preserving Fourier spectral representations. We derive an exact, closed-form variational lower bound that fuses this joint prior into the decoder and optimizes the entire network end-to-end via an alternating EM formulation, avoiding heuristic two-stage clustering while ensuring superior training stability and robustness. Evaluated on OASIS-3 brain MRI, Dual-VaDE delivers substantially sharper and more plausible prototypes, achieves superior cluster separation with improvements of up to 94% over spatial-only baselines, exhibits markedly reduced variance across random seeds, and incurs negligible inference overhead (0.28 ms per image).

0.24

Representation Learning for 3D Brain Imaging: A Benchmark

cs.CV ·2026-11-04 ·Pierre Falconnier, Robin Trombetta, Nicolas Duchateau, Carole Lartizien
top venuebenchmarkcode/data similarity 0.64
Abstract

Many deep learning architectures for computer vision tasks rely on features extracted by a pre-trained encoder on large databases of natural 2D RGB images. However, the wide domain gap with medical images hampers the direct transfer of optimized deep learning models from computer vision to clinical applications. Recently, self-supervised pre-training and foundation models emerged as promising alternatives to mitigate these limitations. Yet, constructing such models for medical imaging faces critical challenges, including designing and adapting training strategies for large 3D volume datasets, and the need for standardized benchmarks to evaluate feature extraction performance. In this study, we propose an extensive comparison of vision encoders with a focus on 3D brain MRI. First, we consider 8 fully and self-supervised models pretrained on natural images (e.g. ResNet, DINO, CLIP) and 10 backbone architectures trained on medical images, either based on supervised (e.g. RadImageNet) or self-supervised (e.g. BiomedCLIP, SimCLR) tasks. All pre-trained encoders are evaluated on various downstream tasks to assess the representation power of their latent representations, including age and sex prediction as well as disease classification on the ADNI and PPMI datasets. Our study shows that 2D encoders pre-trained on natural images perform the most favorably, even compared to models pre-trained on more than 100k brain MRI volumes, highlighting the need for more suitable and robust methods for representation learning in 3D medical imaging. The source code for this study is publicly available at: https: //github.com/PierreFalconnier/brain-representation-benchmark.

0.50

Latent Prediction Yields Cross-Subject Alignable Representations in Sensory-Evoked fMRI

cs.LG cs.AIq-bio.NC ·2026-10-29 ·Serwan Jassim, Victoria Bosch, Daniel Anthes, Elia Bruni, Tim Kietzmann
similarity 0.64
Abstract

Decoding sensory-evoked functional magnetic resonance imaging (fMRI) data is a central analytical tool in cognitive neuroscience. Here we ask whether self-supervised learning (SSL), which has been scarcely explored in this domain, can yield representations that support strong downstream performance. To do so, we adapt a joint embedding predictive architecture (JEPA) to train participant-specific encoders on brain data alone. The encoders never see the stimulus images or any representation derived from them. Applied to the Natural Scenes Dataset (NSD), we evaluate our learned brain embeddings on the downstream tasks of brain captioning and image reconstruction, as well as cross-participant generalization. Our self-supervised approach achieves competitive performance with parameter-rich supervised pipelines that are trained end-to-end with access to image and language data. We further find that encoders trained independently for different participants can be aligned fully unsupervised, using neither images nor stimuli shared between them. After alignment, a neural decoding head trained on one participant can be applied to another without fine-tuning and retains most of its within-subject performance on image reconstruction and brain captioning. By removing the need for images and shared stimuli, our approach is a step towards pooling fMRI data across participants and, eventually, across datasets.

0.12

Evaluating the Generalization of Neuroimaging Foundation Models on African Brain MRI

cs.CV ·2026-09-21 ·Oluwatobi Iyanuoluwa Akinmuleya, Olatokun Shamsudeen Akano, Samuel Danquah Ankapong, Olamide Lawal, Toufiq Musah
published similarity 0.63
Abstract · MICCAI AFRICAI Workshop, Strasbourg, France, 2026, 10 Pages, 1 Figure, 2 Tables

Neuroimaging foundation models pretrained on large, predominantly western cohorts are increasingly proposed as general-purpose backbones for brain MRI analysis. Yet, their ability to generalize to underrepresented clinical populations remains largely untested. We evaluate four recent foundation models (BrainIAC, Neuro-JEPA, NeuroVFM, and Primus) on a three-way diagnostic classification task (Control, Dementia, Parkinson's disease) using a cohort of 88 subjects from a Nigerian clinical brain MRI dataset, across four modality configurations (T1w, T2w, T1w+T2w, FLAIR), and compare against an end-to-end trained ViT3D baseline. The frozen backbones collapse to majority-class predictions, while Neuro-JEPA on FLAIR shows modest but still limited discrimination. In contrast, the end-to-end trained ViT3D achieves higher accuracy and MCC on every task (up to 53.4% accuracy, MCC=0.27) and is the only model with non-trivial recall. Our findings suggest that these frozen neuroimaging foundation models are insufficient for fine-grained diagnostic classification in small, non-western clinical cohorts, motivating parameter-efficient adaptation and broader multi-site external validation for equitable deployment in global health settings.

0.29