CS Paper Radar

Bridging Modalities on the Cortex: Surface-based MRI to PET Translation with a Diffusion Bridge

Yitong Li, Alexandra Samoylova, Fabian Bongratz, Timo Grimmer, Dennis M. Hedderich, Igor Yakushev, Christian Wachinger

cs.CV cs.AI ·submitted 2026-09-17 12:38 UTC · v1
code/data

Abstract

Cortical hypometabolism measured by Fluorodeoxyglucose Positron Emission Tomography (FDG-PET) is a highly sensitive biomarker for dementia diagnosis. However, high costs, radiation exposure, and limited accessibility constrain its clinical utility. While cross-modal synthesis from Magnetic Resonance Imaging (MRI) offers a promising alternative, existing volumetric generation methods do not explicitly account for the highly folded cortical geometry, where disease-related patterns predominantly reside. To address this, we introduce a novel surface-based diffusion bridge framework DB-SUiT for MRI-to-PET translation that operates natively on the cortical manifold. A conditional Spherical U-shaped vision Transformer (SUiT) is specifically designed to model the intricate cross-modal relationships while preserving surface topology. It combines spherical convolutional encoders for multi-scale surface feature extraction with bottleneck Transformers to capture long-range spatial dependencies, while incorporating demographic and subcortical conditions to refine the synthesis. Evaluated on two datasets, including subjects with different dementia types, DB-SUiT demonstrates high-fidelity synthesis that substantially outperforms other baselines. In automated dementia classification, synthesized PET surfaces improve performance over MRI by 14.2% and PET volumes by 11.3%, approaching the performance of real PET surfaces. In a blinded reader study, synthetic PET achieved 85.5% diagnostic accuracy, compared with 75.8% for MRI and 95.2% for real PET. This further demonstrates cross-cohort and cross-pathology generalization, as the model was evaluated without retraining on an external cohort that included a dementia subtype not represented during training. Our code is available at https://github.com/ai-med/DB-SUiT.

Closest papers

Nearest neighbours by meaning across the whole index.

Mr3D-VL: A generalist vision language foundation model for Multiparametric 3D Magnetic Resonance Imaging

cs.CV cs.AI ·2026-08-13 ·Zhi Qiao, Xintong Wu, Yichu He, Feng Shi
similarity 0.67
Abstract

Multi-parametric magnetic resonance imaging (mpMRI) is a cornerstone for brain tumor diagnosis and treatment, yet current AI models face critical limitations: their lack of natural language interaction and interpretability impedes spatial information integration and cross-modal reasoning required clinically. Key challenges arise from significant physical meaning differences across modalities, spatial misalignment due to scan intervals, and the need for complex multi-feature interpretation in tasks like glioma grading. While visual-language models (VLMs) show promise in cross-modal understanding, existing methods focus mainly on 2D image modeling, neglecting direct perception of 3D volumetric space. Although 3D VLMs have been proposed for report generation and feature alignment in 3D CT imaging, mpMRI applications demand collaborative inference across multiple imaging modalities-a requirement unmet by current solutions. To address this, we introduce Mr3D-VL, a dedicated visual-language foundation model for multi-parametric 3D MRI. With 4 billion parameters, it employs an unsupervised pre-trained shared 3D encoder and 4D rotational positional embedding for dual modality-spatial integration. Its cross-modal projection layer uses a multi-resolution feature implantation strategy to enhance feature perception across resolutions. Experimental results show significant improvements over existing 4B/7B/30B domain-specific and general-purpose models in text generation tasks, achieving a BERTScore of 0.856 for report generation, with question-answering accuracy at 0.713 and multiple-choice accuracy at 0.912.

0.04

Reduce, Then Encode: Multiscale Volumetric Reduction for 2D Foundation Models in Brain MRI

cs.CV ·2026-09-28 ·Dexuan Ding, Yuankai Qi, Bogong Wang, Luping Zhou, Amin Beheshti
similarity 0.67
Abstract

Pretrained 2D foundation models offer a practical alternative to dedicated 3D pretraining for brain structural magnetic resonance imaging (sMRI), but their use on volumetric data requires bridging the mismatch between a 2D encoder and a 3D volume input. Existing methods typically encode slices independently and integrate their features afterwards. We introduce Multiscale Volumetric Reduction (MVR), a reduce-then-encode approach that compresses each anatomical view from (D) slices into (M << D) complementary 2D components before foundation-model encoding. MVR combines an uncentered-PCA base component derived from the original through-plane intensities with residual detail components constructed from multiscale spatial descriptors. The reduction is estimated from the training volumes without diagnostic labels or gradient-based optimization and remains fixed thereafter. The resulting components are independently processed by a shared frozen 2D foundation model and concatenated for linear probing. Under this frozen-encoder setting, MVR achieves strong overall performance across ADNI, OASIS, and ABIDE relative to the evaluated 2D-to-3D adaptation methods and simple input-reduction baselines, while also generalizing strongly from ADNI to AIBL.

0.13

3D CT-to-PET Translation via Latent Brownian Bridge Diffusion

cs.CV cs.AI ·2026-09-11 ·Sarita Mourya, Francesco Di Feola, Pierangelo Veltri, Paolo Soda
similarity 0.67
Abstract

Computed tomography (CT) and positron emission tomography (PET) provide complementary anatomical and functional information for cancer diagnosis and treatment planning. However, the widespread use of PET is limited by high radiation exposure, elevated costs, and restricted availability. To address these limitations, deep learning-based CT-to-PET translation has emerged as a promising approach for synthesizing PET-like information directly from CT images, although accurately modeling the large cross-modal gap remains challenging. In this work, we propose a 3D CT-to-PET translation framework based on latent Brownian Bridge Diffusion (BBDM). The method consists of two stages. First, a Variational Autoencoder (VAE) is trained on paired CT-PET patches, integrating contrastive learning to improve latent alignment between anatomical and metabolic representations. Second, a BBDM is trained in the latent space to translate CT latent representations into their corresponding PET counterparts. The translated PET latents are then decoded and stitched to reconstruct the final 3D PET volume. We evaluate the proposed approach on two publicly available datasets. Quantitative results based on image fidelity and lesion-level PET-specific metrics demonstrate improved performance compared with competing methods. In particular, the proposed approach improves PET signal fidelity, better preserves clinically relevant uptake patterns, and shows improved performance in preserving small-lesion metabolic activation, paving the way for virtual imaging applications.

0.22

Uncertainty-Aware Cortical Surface Reconstruction and Spherical Registration for Heterogeneous Brain MRI

eess.IV cs.CV ·2026-10-12 ·Jesper Duemose Nielsen, Karthik Gopinath, Andrew Hoopes, Adrian Dalca, Axel Thielscher +2 more
similarity 0.65
Abstract

Surface-based cortical analysis is useful in many applications as it is a natural way to model the highly folded structure of the cerebral cortex, allowing quantities such as curvature and thickness to be easily calculated. Existing deep learning (DL) methods have focused on 1x1x1 mm3 T1-weighted (T1w) scans, thus replicating classical pipelines. Here we present a neural network that is trained exclusively on synthetic scans with random contrast and resolution. Consequently, it can be applied to almost any kind of magnetic resonance imaging (MRI) scan. In addition, our model provides vertex-wise uncertainty estimates of the predicted surface as well as a spherical registration to fsaverage space. Our model performs on par with other state-of-the-art DL methods on 1x1x1 mm3 T1w scans and outperforms the only other method capable of estimating cortical surfaces from low resolution, non-T1w scans. The predicted uncertainty, moreover, correlates with overall scan quality on low resolution scans and highlights areas of poor surface reconstruction on 1x1x1 mm3 T1w scans, suggesting that it can be used for quality control. Finally, the predicted spherical registrations prove comparable to other DL and classical registration methods. These qualities, when combined with the fast inference time, will provide the ability to analyze large, existing databases of clinical MRI scans, with the potential for finding disease-specific patterns that would otherwise be infeasible in a research setting.

0.29

DTI-Guided Volumetric Spherical Harmonics Regression for Single-to-Multi-Shell dMRI Synthesis

cs.CV ·2026-09-13 ·Binghua Li, Christina Andica, Tong Liang, Ziqing Chang, Chao Li +6 more
top venuecode/data similarity 0.65
Abstract · 10 pages, 4 figures. Accepted at MICCAI 2026

Multi-shell diffusion MRI (dMRI) unlocks more expressive microstructural modeling than single-shell scans, yet its longer acquisition time hinders deployment in large-scale cohorts and time-constrained clinical settings. Synthesizing an unobserved shell from a single-shell input is fundamentally ill-posed and further complicated by protocol mismatch, where source and target gradient direction sets may not align. We propose DTI-SHNet, a single-to-multi-shell synthesis framework that operates in the real symmetric spherical harmonics (SH) coefficient domain and performs spatially aware volumetric regression. Given a source shell, we estimate diffusion tensor imaging (DTI) and use direction-agnostic parametric maps along with a brain mask as conditioning priors to guide a 3D U-Net regressor from source-shell to target-shell SH coefficients. To couple coefficient accuracy with signal fidelity, we introduce a signal consistency regularization that reconstructs signals on randomly sampled canonical directions from predicted coefficients and enforces agreement in the signal domain. Experiments on UK Biobank and Cam-CAN data for b=1000 to b=2000 dMRI synthesis show that DTI-SHNet achieves competitive visual quality compared to advanced methods, while better preserving downstream diffusion measures. Our code is available at https://github.com/xiaovhua/dti-shnet.

0.51

Accurate MRI-free PET Quantification and Reliability Assessment via CT-to-MRI Synthesis and Conformal Prediction

cs.CV ·2026-10-29 ·Jungwoo Seo, Seung Yun Choi, Kun Ho Lee, Seun Jeon, Byoung Seok Ye +1 more
similarity 0.64
Abstract · 28 pages, 8 figures, including supplementary material. Submitted to Medical Image Analysis

Standardized uptake value ratio (SUVR) quantification of amyloid PET commonly relies on a subject-specific T1-weighted MRI, yet a usable MRI is not always available. We therefore combined CT-to-MRI synthesis with conformal prediction for MRI-free PET quantification with reliability assessment. A slice-consistent Brownian Bridge Diffusion Model was strengthened with multi-dataset training and data-source conditioning to enrich the anatomical prior learned from MRI, and with expanded multi-slice CT context to exploit the through-plane continuity of subject-specific anatomy in CT. Synthetic MRI was used for FreeSurfer-based anatomical parcellation and PET quantification, while deep-ensemble uncertainty was calibrated with conformal prediction to estimate a plausible range for the corresponding real MRI-based SUVR. On $^{11}$C-PiB PET from OASIS-3, the proposed method achieved the highest synthesis and parcellation fidelity among the compared methods. Mean cortical SUVR from the proposed synthetic MRI agreed closely with the real MRI-based reference (MAE $0.016 \pm 0.015$, Pearson $r = 0.9987$) and yielded the same amyloid-positivity classification as the real MRI in all 87 test subjects at the prespecified cut-off. Conformal prediction achieved empirical coverage close to the prespecified levels, and at 90% target coverage, 85 of 87 subjects had prediction intervals that supported an unambiguous amyloid-positivity decision. Zero-shot evaluation on an independent external cohort showed that the proposed CT-to-MRI model could support PET quantification without retraining, although conformal calibration required target-cohort recalibration to recover the intended coverage. These findings support CT-to-MRI-based MRI-free PET quantification as a promising approach for providing accurate SUVR measurements and an explicit assessment of their reliability.

0.31

Evidence Before Accuracy: A MRI-PET Fusion Network for Alzheimer Disease Classification with Causal Regional Validation

cs.CV ·2026-09-28 ·Saeid Firouzi Daghigh, Saeed Ayat
similarity 0.64
Abstract

Deep learning models for Alzheimer disease (AD) classification routinely report near-perfect discrimination, yet few are shown to rest on AD-relevant neurobiology rather than on dataset artifacts, subject-level leakage, or non-brain image content. We present a fusion network combining T1 MRI and FDG PET across axial, coronal, and sagittal planes, trained on ADNI consists of 554 paired subjects. The fusion model reaches AUC 0.962, accuracy 0.909, and F1 0.891, competitive with recent 3D CNN and multimodal transformer systems at substantially lower cost. We first quantify how much modality, plane and slice geometry matter. A validation-only search over slice centres and neighbour spacings moves AUC by 0.180 for MRI and 0.078 for PET, selecting narrow spacing for MRI and wide spacing for PET, with the chosen coronal centres falling on the hippocampal body and on the posterior cingulate respectively. The contribution, however, is the evidence layer built around that number. Shortcut controls collapse the model to AUC 0.622 (silhouette), 0.608 (exterior), and 0.500 (blank), and a label-permutation null yields 0.456. Forward region-of-interest (ROI) ablation shows that masking medial temporal cortex in MRI and the posterior default-mode network (DMN) in PET produces the largest shift in the AD logit, while area-matched controls remain indistinguishable from that null. Reverse ROI ablation shows that the medial temporal lobe alone retains 89.2% of above-chance discrimination in MRI and the posterior DMN alone retains 79.0% in PET. A quantitative comparison of attribution methods shows occlusion sensitivity reaching 3.5-5.0* enrichment inside a priori AD regions against 0.10-0.43* in controls. Ablation and attribution independently establish a biologically correct double dissociation: hippocampal evidence is carried by MRI, posterior cingulate evidence by PET.

0.12

Unified Multi-plane Autoregressive Diffusion for 3D Multi-contrast MRI Synthesis

cs.CV ·2026-10-07 ·Yejee Shin, Geonhui Son, Jinglu Wang, Minwoo Jung, Yan Lu +1 more
top venue similarity 0.64
Abstract · Accepted to ECCV 2026

Acquiring a complete set of magnetic resonance imaging (MRI) contrasts is time-intensive and uncomfortable for patients, despite the diagnostic value of multi-contrast imaging. This motivates synthesizing missing contrasts from those already acquired, which is an inherently 3D problem requiring anatomical coherence across axial, sagittal, and coronal planes. However, fully 3D generative models are often impracti- cal under computational resources that scale cubically with volume size. We propose a unified Multi-Plane Autoregressive Diffusion (MPAD), a latent diffusion framework that achieves full-volume 3D synthesis using efficient plane-wise 2D operations while preserving volumetric coherence. A 3D autoencoder first compresses MRI scans into an isotropic 3D la- tent representation. A 2D diffusion model is then trained to reconstruct masked latent slices of the target contrast, conditioned on both source- contrast slices and unmasked target-contrast slices. During inference, we introduce plane-wise autoregressive synthesis with inter-plane priors. Slices are generated autoregressively in random order within one plane orientation to maintain intra-plane continuity, then propagated as con- ditioning priors to orthogonal plane orientations to enforce inter-plane consistency. Compared to 3D latent diffusion baselines, MPAD reduces training and inference FLOPs by 7x and 3x, respectively, while also lowering inference time and peak memory consumption. Experiments on multiple datasets demonstrate that MPAD achieves superior perfor- mance, generating high-fidelity 3D volumes and supporting one-to-many translation within a single unified model.

0.33