CS Paper Radar

Unified Multi-plane Autoregressive Diffusion for 3D Multi-contrast MRI Synthesis

Yejee Shin, Geonhui Son, Jinglu Wang, Minwoo Jung, Yan Lu, Dosik Hwang

cs.CV ·submitted 2026-10-07 03:31 UTC · v1
top venue

Abstract

Acquiring a complete set of magnetic resonance imaging (MRI) contrasts is time-intensive and uncomfortable for patients, despite the diagnostic value of multi-contrast imaging. This motivates synthesizing missing contrasts from those already acquired, which is an inherently 3D problem requiring anatomical coherence across axial, sagittal, and coronal planes. However, fully 3D generative models are often impracti- cal under computational resources that scale cubically with volume size. We propose a unified Multi-Plane Autoregressive Diffusion (MPAD), a latent diffusion framework that achieves full-volume 3D synthesis using efficient plane-wise 2D operations while preserving volumetric coherence. A 3D autoencoder first compresses MRI scans into an isotropic 3D la- tent representation. A 2D diffusion model is then trained to reconstruct masked latent slices of the target contrast, conditioned on both source- contrast slices and unmasked target-contrast slices. During inference, we introduce plane-wise autoregressive synthesis with inter-plane priors. Slices are generated autoregressively in random order within one plane orientation to maintain intra-plane continuity, then propagated as con- ditioning priors to orthogonal plane orientations to enforce inter-plane consistency. Compared to 3D latent diffusion baselines, MPAD reduces training and inference FLOPs by 7x and 3x, respectively, while also lowering inference time and peak memory consumption. Experiments on multiple datasets demonstrate that MPAD achieves superior perfor- mance, generating high-fidelity 3D volumes and supporting one-to-many translation within a single unified model.

Comment
Accepted to ECCV 2026
Journal ref
ECCV 2026. Lecture Notes in Computer Science, vol 17034. Springer, Cham
DOI
10.1007/978-3-032-37450-9_16

Closest papers

Nearest neighbours by meaning across the whole index.

3D Diffusion Models for Medical Imaging

cs.CV ·2026-10-19 ·Frank-Peter Schilling, Daniel Barco, Philipp Denzel, Marc Stadelmann
publishedsurvey similarity 0.73
Abstract · 27 pages, accepted for publication in Informatics in Medicine Unlocked

Generative deep learning methods such as Variational Autoencoders, Generative Adversarial Networks or Diffusion Models have achieved remarkable results in image synthesis. In particular, Diffusion Models have demonstrated superior image fidelity and sample diversity compared to other approaches on standard benchmarks, while also offering more flexible training dynamics as evidenced both in natural image domains and in medical applications. In medical imaging including Computed Tomography, Magnetic Resonance Imaging or Positron Emission Tomography, such models have various important use-cases including data augmentation, domain transfer, image reconstruction, inpainting, image augmentation and artifact correction. Further, generated images can be conditioned on patient characteristics, medical reports, segmentation masks or other images. Specific challenges arise due to the 3D volumetric nature of medical images, which translate to substantial demands in terms of memory, compute, and training data. This article provides a comparative overview of 3D Diffusion Models which have been developed for medical images, with a focus on current performance, limitations, and an outlook for future work.

0.23

High-Resolution Quantitative MRI Using Contrast-Domain Multiscale Energy (MuSE) Regularization

eess.IV ·2026-11-03 ·Rushdi Rusho, Jyothi Rikhab Chand, Steve R. Kecskemeti, James H. Holmes, Mathews Jacob
similarity 0.73
Abstract · 5 pages, 3 figures

Quantitative MRI methods can offer tissue parameter maps (e.g. T1) that are comparable across scanners, sessions and subjects, together with multiple contrast-weighted images from a single scan. However, most data are acquired at 1 mm, which limits their use in neurodegenerative diseases that affect fine brain structures. We introduce, to our knowledge, the first diffusion-model-based reconstruction for quantitative MRI at 0.5 mm isotropic resolution from scans of less than 5 minutes, which has eightfold smaller voxel volume than clinical protocols. The key idea is to apply the prior to the synthesized contrast images rather than to the subspace coefficients. The prior is then independent of the temporal basis and the sequence, which improves generalization and allows a 2-D multiscale energy (MuSE) model trained on grayscale CelebA face images to be used without any MRI training data. The explicit energy makes reconstruction MAP estimation by a step-size-free majorize-minimize algorithm that regularizes a rotating subset of contrasts along different axes, which makes 3-D multi-contrast reconstruction tractable. For 3-D radial MPnRAGE at 7 T in 4.3 min, MuSE reduced the phantom NRMSE from 0.133 (CG-SENSE) and 0.091 (L1-wavelet) to 0.074 and the WM T1 interquartile range from 117 and 47 ms to 38 ms, and it was demonstrated in vivo at 7 T.

0.24

SciFusion: Physics-Guided Compositional Diffusion for Multimodal 3D Scientific Image Generation

eess.IV cs.LG ·2026-10-21 ·Luoxi Wang, Alessio Quercia, Hanno Scharr, Arya Bangun
similarity 0.71
Abstract

Existing 3D generative approaches are typically developed for individual imaging modalities, requiring substantial modality-specific data and limiting their ability to exploit shared structure across scientific imaging domains. We present SciFusion, a multimodal framework for unified 3D scientific image generation built on a latent diffusion backbone, with two key contributions. First, we introduce a compositional conditioning scheme that factorizes each modality into domain, object, and contrast representations. Sharing these factors across related modalities enables knowledge transfer across heterogeneous imaging domains and allows the factors to be recombined at inference time to synthesize unseen domain-object-contrast compositions in a zero-shot manner. Second, we incorporate domain-specific physics-informed objectives: Fourier spectral consistency for MRI, Radon projection consistency for CT, and multislice diffraction consistency for electron microscopy, to encourage generated samples to respect modality-dependent image formation processes. Extensive experiments across 15 medical and scientific imaging modalities demonstrate that SciFusion improves generation fidelity and cross-domain generalization over modality-specific and flattened multimodal conditioning strategies, while enabling zero-shot synthesis of unseen compositions without training on those target compositions. These results demonstrate that compositional, physics-guided conditioning provides a scalable framework for unified 3D scientific image generation, particularly for emerging modalities where large-scale training data are difficult to obtain.

0.19

Multimodal-Conditioned Latent Diffusion Models for Spatiotemporal Brain MRI Forecasting

cs.CV cs.AI ·2026-10-18 ·Antonio Scardace, Silvia Ingala, Sebastiano Battiato, Daniele Ravì
publishedcode/data similarity 0.70
Abstract · Accepted to WACV 2027

Spatiotemporal forecasting of 3D anatomical changes is a critical generative vision challenge for modeling Alzheimer's Disease (AD) progression. While brain MRI forecasting from single-modality data has advanced, other key complementary multimodal biomarkers remain underutilized. In this work, we propose the Multimodal-Conditioned Latent Diffusion Model (MC-LDM), a novel framework for patient-specific brain MRI synthesis. MC-LDM leverages multimodal biomarkers---MRI, tau-PET, Clinical Dementia Rating (CDR), and APOE4---during training, but relies only on baseline MRI, APOE4, and patient metadata at inference, reflecting realistic clinical availability. Our contributions are threefold: (1) a Latent Diffusion Model (LDM) fine-tuned via Low-Rank Adaptation (LoRA) for data-efficient learning; (2) a multimodal conditioning that integrates molecular and clinical signals to model heterogeneous disease trajectories; and (3) auxiliary predictors estimating unobserved multimodal biomarkers from MRI features and patient covariates to condition the diffusion model, removing the need for these assessments at inference. Trained on 1,355 visits from 852 subjects in the Alzheimer's Disease Neuroimaging Initiative (ADNI), MC-LDM reduces average regional volumetric MAE by 18% over the second-best baseline on an independent test set of 664 visits. To our knowledge, MC-LDM represents the first diffusion-based framework to leverage multimodal priors for MRI forecasting. This conceptual advancement enables patient-specific trajectory generation that captures aspects of AD heterogeneity not observable from structural anatomy alone, bridging the gap between rich multimodal supervision and the strict constraints of routine clinical deployment. Code is available at: https://github.com/antonioscardace/MC-LDM.

0.43

Volumetric Latent Diffusion for Brain Tumour Segmentation with Missing Modalities

eess.IV ·2026-11-05 ·Toan Nguyen Hai, Hung Nguyen Tuan, Ba Luong Son, Huynh Trinh Ngoc
similarity 0.70
Abstract

Brain tumour segmentation from magnetic resonance imaging normally assumes that all four imaging sequences are available, but in clinical practice one or more are frequently missing. Existing methods for incomplete inputs learn modality-invariant representations or distil from a full-modality teacher, and return a single deterministic mask. Gen- erative formulations instead model a distribution over plausible masks, but the cost of sampling at full resolution has largely confined them to two dimensions. This paper proposes a three-dimensional conditional latent diffusion framework for whole-tumour segmentation from incom- plete multi-sequence MRI. Images and annotations are first compressed by two volumetric autoencoders, which makes diffusion over whole vol- umes affordable, and sequences are dropped at random during training so that a single model handles every combination of missing sequences. Because the model samples rather than predicts, repeated draws also yield a voxel-wise map of disagreement at no extra training cost. Evalu- ations on 2,601 studies from BraTS 2023 and 2024 show that the method reaches 0.955 Dice with all four sequences and 0.904 with FLAIR alone, and that it is the most accurate of the four methods compared in eight of the fifteen configurations. The disagreement between draws widens as sequences are withheld, giving a signal that accompanies the prediction rather than replacing it

0.21

AnaDiffusion: Anatomically Compositional Latent Diffusion for Controllable 3D Brain MRI Generation

cs.CV ·2026-08-24 ·Huiwen Han, Lulin Liu, Bangya Liu, Yuanhao Cai, Nuo Chen +6 more
similarity 0.69
Abstract

3D brain MRI generation has made significant advances in medical imaging, simulation, and controllable anatomical analysis. However, existing generative models typically synthesize 3D volumes monolithically, often overlooking regional anatomical structures and limiting local controllability. To address these limitations, we introduce AnaDiffusion, an anatomically compositional latent diffusion framework that factorizes the generation process into distinct, anatomically meaningful regions, followed by part-to-whole assembly and global refinement. Our approach first trains part diffusion models to capture local structural priors. We then inject an assembled anatomical composite of the parts into the whole-brain latent representation and continue denoising. This mechanism enables the model to resolve global context while preserving the injected anatomy. As a result, AnaDiffusion produces both explicit part assets and a globally coherent volume, thereby enabling controllable part editing without requiring subject-specific dense segmentation maps at inference time while maintaining consistent part-to-whole brain structure. On the subject-disjoint ADNI test split, AnaDiffusion achieves the lowest FID across the whole brain, left and right hemispheres, cerebellar-brainstem complex, and seam regions. It also achieves the best cerebellar and second-best ventricular and brainstem absolute Cohen's d values among the evaluated methods. In localized editing experiments, paired MS-SSIM demonstrates high target transfer and off-target preservation, supporting controllable part replacement with minimal unintended anatomical alterations.

0.24

Latent Space Is Not Flat: Rethinking Latent Structure for 3D Medical Image Synthesis

cs.CV ·2026-09-26 ·Haowen Xue, Hao Chen, Hexuan Hu, Qian Huang, Yi Han +4 more
similarity 0.68
Abstract · 5 pages, 4 figures, 4 tables

Latent generative models make 3D medical image synthesis computationally practical by generating in a compressed space. However, we show that the common flat Euclidean assumption induced by $\ell_2$ objectives is imprecise: latent-space geometry is so strongly anisotropic that equal-magnitude errors can produce drastically different decoded distortions. We further find that this anisotropy has a clear feature: sensitive variation concentrates in a low-rank subspace. The dominant low-rank components capture the overall structure, encoding long-range, spatially coordinated variation while remaining resistant to local noise. Its orthogonal residual, in contrast, mainly captures local and image-specific variation. Motivated by this asymmetry, we introduce Latent Structure Flow (LSF). At each block, LSF decomposes the latent state into structure and residual, models structural changes with global context, and predicts residual variation locally while preserving a direct path for the input structure. LSF changes only the generator, leaving the frozen codec and pointwise training objective unchanged. Across cross-modality synthesis and tumor inpainting tasks, LSF outperforms all compared baselines on both global and tumor-specific metrics, demonstrating the benefit of explicitly modeling latent-space structure for 3D medical image synthesis.

0.19

Diffusion-Based Synthesis of Complete 3D Biparametric Prostate MRI

cs.CV ·2026-10-22 ·Claudia Giardina, Montse Pardàs, Verónica Vilaplana
publishedhot direction similarity 0.68
Abstract · Presented at the 6th Deep Generative Models Workshop (DGM4MICCAI), MICCAI 2026

Synthetic biparametric MRI (bpMRI) may help alleviate the limited availability of large and representative prostate MRI datasets for deep learning applications. In this work, we present a conditional 3D diffusion framework for anatomy-conditioned synthesis of complete prostate bpMRI from anatomical masks. The framework supports sequential synthesis, in which T2w and diffusion modalities are generated by separately trained models, and joint synthesis, in which all bpMRI modalities are generated simultaneously within a single model. Five configurations were trained using PI-CAI, FastMRI Prostate, and PROSTATEx datasets to investigate different conditioning schemes, target modalities, and synthesis strategies. Evaluation using distribution-based metrics, paired-image similarity, and qualitative assessment demonstrated the feasibility of both synthesis pathways. Joint synthesis achieved competitive distributional fidelity while generating complete bpMRI, whereas direct synthesis of ADC and computed high-b-value DWI outperformed their derivation from synthetic diffusion-weighted images.

0.34