CS Paper Radar

Clinical Pathways Matter for Multimodal Deep Learning in Early Alzheimers Disease Detection

Yao Lu, Solveig Kristina Hammonds, Alvaro Fernandez-Quilez

eess.IV ·submitted 2026-08-17 06:17 UTC · v1
published

Abstract

Identifying individuals at risk of Alzheimer's disease (AD), particularly in the preclinical and early stages, remains challenging. Although deep learning approaches based on structural MRI show promise as a non-invasive biomarker, existing multimodal models require task-specific training and depend on biomarkers that are not routinely available in clinical practice. Here, we propose a zero-shot multimodal framework based on SigLIP that combines structural MRI embeddings with text embeddings of routinely collected clinical variables for early AD risk stratification in individuals at preclinical or mild cognitive impairment (MCI) stages. We evaluated the approach in 416 individuals from the ADNI cohort (age: 72.73 +- 6.7). SigLIP was used without fine-tuning to extract MRI and clinical text embeddings, which were combined into multimodal representations for individual-level AD risk prediction within 4 years. We further compared the model performance in a single-visit and two-visit settings to assess the value of longitudinal information and framework scalability. In the single-visit setting, combining MRI embeddings with MMSE, age, and sex achieved an AUC of 0.91 +- 0.02, outperforming both a CSF A\b{eta}42-based model (AUC 0.73 +- 0.08) and an MMSE-based model (AUC 0.85 +- 0.22). In the two-visit setting, performance was maintained or improved, supporting the scalability of the approach to longitudinal data. These findings suggest that zero-shot multimodal fusion of structural MRI and routinely collected clinical variables may provide a practical and scalable strategy for early AD risk stratification without task-specific retraining.

Comment
Published
DOI
10.1038/s41598-026-57861-z

Closest papers

Nearest neighbours by meaning across the whole index.

A Multimodal Explainable Deep Learning Framework for Alzheimer's Disease Diagnosis using 3D Magnetic Resonance Imaging and Clinical Data

cs.CV stat.ML ·2026-09-11 ·Yusuf Brima, Marcellin Atemkeng, Lakshmana Rao Namamula, Antoine Vacavant
similarity 0.77
Abstract

Dementia is a major and growing global health burden, with Alzheimer's disease (AD) accounting for most cases. Timely and accurate diagnosis is central to managing this burden and increasingly depends on integrating complementary clinical and imaging information. Multimodal deep learning can combine these modalities for AD diagnosis, but how its explanations behave across modalities, fusion strategies, and cohorts remains unclear. We developed an explainable multimodal framework pairing a 3D CNN encoder for T1-weighted MRI with a feedforward network for harmonized clinical and demographic data, comparing varied model setups on three-way and pairwise diagnostic tasks using 6,479 internal records from the ADNI and 1,703 independent records from the OASIS-3. On ADNI, the tabular-only model achieved the highest three-class AUC-ROC of 0.879 and best discriminated cognitively normal (CN) versus mild cognitive impairment (MCI; 0.903), while cross-attention performed best for MCI versus AD (0.861); CN versus AD was highly discriminative overall. On OASIS-3, the vision-only model performed best (three-class AUC-ROC 0.910); CN versus MCI remained difficult, and no fusion strategy consistently outperformed single modalities across tasks and cohorts. SHAP and Integrated Gradients identified the MMSE as the dominant tabular feature in both cohorts, with global feature rankings agreeing strongly in ADNI ($ρ=0.94$) and OASIS-3 ($ρ=0.96$); CAM-based explanations, however, changed with model configuration and cohort. These findings show that multimodal performance and explanations are task, modality, fusion, and cohort-dependent: a dominant cognitive signal persisted across cohorts, but feature contributions and CAM explanations did not, underscoring the need to evaluate explainability under cohort shift rather than as a stable, intrinsic property.

0.10

Multimodal-Conditioned Latent Diffusion Models for Spatiotemporal Brain MRI Forecasting

cs.CV cs.AI ·2026-10-18 ·Antonio Scardace, Silvia Ingala, Sebastiano Battiato, Daniele Ravì
publishedcode/data similarity 0.71
Abstract · Accepted to WACV 2027

Spatiotemporal forecasting of 3D anatomical changes is a critical generative vision challenge for modeling Alzheimer's Disease (AD) progression. While brain MRI forecasting from single-modality data has advanced, other key complementary multimodal biomarkers remain underutilized. In this work, we propose the Multimodal-Conditioned Latent Diffusion Model (MC-LDM), a novel framework for patient-specific brain MRI synthesis. MC-LDM leverages multimodal biomarkers---MRI, tau-PET, Clinical Dementia Rating (CDR), and APOE4---during training, but relies only on baseline MRI, APOE4, and patient metadata at inference, reflecting realistic clinical availability. Our contributions are threefold: (1) a Latent Diffusion Model (LDM) fine-tuned via Low-Rank Adaptation (LoRA) for data-efficient learning; (2) a multimodal conditioning that integrates molecular and clinical signals to model heterogeneous disease trajectories; and (3) auxiliary predictors estimating unobserved multimodal biomarkers from MRI features and patient covariates to condition the diffusion model, removing the need for these assessments at inference. Trained on 1,355 visits from 852 subjects in the Alzheimer's Disease Neuroimaging Initiative (ADNI), MC-LDM reduces average regional volumetric MAE by 18% over the second-best baseline on an independent test set of 664 visits. To our knowledge, MC-LDM represents the first diffusion-based framework to leverage multimodal priors for MRI forecasting. This conceptual advancement enables patient-specific trajectory generation that captures aspects of AD heterogeneity not observable from structural anatomy alone, bridging the gap between rich multimodal supervision and the strict constraints of routine clinical deployment. Code is available at: https://github.com/antonioscardace/MC-LDM.

0.43

MMAP: Multimodal Missing-Aware Pretraining for Longitudinal Alzheimer's Prediction

cs.CV cs.LG ·2026-09-22 ·Fiona Kekwick, Matthew Baugh, Bernhard Kainz, Paul M. Matthews, Wenjia Bai
published similarity 0.71
Abstract · To be published in the proceedings of the 2026 MICCAI Workshop on Multimodal Learning with Medical Tabular Data

Clinical decision making heavily relies on predicting the disease progression trajectory by seeking to understand patient's health status which is characterised by multimodal medical data. AI holds great potential for learning useful representations from multimodal medical data to predict disease progression and aid clinical decision making. However, development of predictive AI models is constrained by missing modalities and incomplete tabular data frequently occurring in medical datasets. In addition, disease labels alone may only provide limited supervisory signals for learning representations from high-dimensional multimodal data. Here, we present MMAP, a novel Multimodal Missing-aware Alignment Pretraining method for learning image-tabular representations from incomplete data. An image encoder is pretrained with efficient sigmoid contrastive learning combined with generative reconstruction. A tabular encoder is built upon a tabular foundation model. A missing token generator enables the two encoders to take incomplete data as input, enabling the model to be robust against missing modalities, either with missing images or missing tabular data. We evaluate the clinical usefulness of the learnt multimodal representations on two challenging longitudinal clinical tasks for Alzheimer's disease: predicting disease stage conversion and predicting amyloid status. The proposed method outperforms strong multimodal and unimodal baselines.

0.26

Leakage-Controlled Multimodal Learning for Diagnosis and Progression Prediction in Alzheimer's Disease Research

cs.LG ·2026-10-07 ·Akeem Temitope Otapo, Ghazaleh Khodabandelou, Zuheng Ming, Alice Othmani
similarity 0.70
Abstract

Alzheimer's disease prediction involves irregular visits, heterogeneous measurements and incomplete modalities. This study presents a multimodal multitask framework combining an adapted SFCN MRI encoder, four causal clinical Transformers, shared fusion and task-specific ODE-GRU dynamics. Fine-tuning and LoRA adapt the final two MRI blocks. Task-DRO balances task losses, while Group-CVaR targets cohort and comorbidity strata. Branch-specific input controls, subject-grouped partitions and empirical causality checks support longitudinal evaluation. Across 2,649 subjects and 17,317 visits from ADNI, OASIS-2 and MIRIAD, internal validation yields diagnosis, stage-1 progression and first-stage-1-visit progression AUROCs of 0.935 +/- 0.002, 0.884 +/- 0.003 and 0.870 +/- 0.005, respectively (mean +/- SD across three seeds). Corresponding hybrid AUROCs are 0.951, 0.909 and 0.896. Next-visit MMSE mean absolute error (MAE) is 1.61 points; worst-stratum diagnosis AUROC is 0.827 +/- 0.008. Sampled ADNI explanations identify task-specific input dependence. OASIS-3 external validation yields network and hybrid diagnosis AUROCs of 0.763 and 0.767, hybrid next-visit progression AUROC of 0.764, diagnosis calibration error decreasing from 0.197 to 0.052, and next-visit MMSE MAE of 0.86. Seed-42 paired ablations of six components yield pooled diagnosis and progression AUROC differences between -0.004 and +0.004; removing clinical encoder inputs lowers diagnosis AUROC by 0.272. The framework integrates longitudinal prediction, missing-modality handling, auxiliary comorbidity modelling and subgroup evaluation within a common pipeline.

0.11

AG-JEPA: Anatomically Guided Joint Embedding Predictive Architecture for Neurodegenerative Representation Learning

cs.CV cs.AI ·2026-11-13 ·Azad Singh, Ayantika Das, Amog Rao, Aiman Farooq, Anirban Porya +5 more
similarity 0.70
Abstract

Brain MRI can reveal early signs of Alzheimers disease (AD) before measurable cognitive decline becomes apparent. However, during the transition from cognitively normal (CN) to mild cognitive impairment (MCI), the disease-related signals are often subtle and confined to only a few brain regions. Existing Joint Embedding Predictive Architecture (JEPA) models learn general structural representations but do not explicitly capture the anatomical regions affected in AD. We propose Anatomically Guided JEPA (AG-JEPA), which introduces anatomical tokens that interact with patch tokens during pretraining to learn region-specific representations from brain regions relevant to neurodegeneration. Building on this representation space, we further propose Prog-AG-JEPA, which models disease progression trajectory as a conditioned latent displacement within this space. We evaluate AG-JEPA across five downstream tasks spanning Alzheimer's disease diagnosis and progression prediction. AG-JEPA improves diagnosis and MCI prognosis, achieving an 8.4 percentage point AUROC gain over the strongest baseline on CN/MCI classification and a 7.1 percentage point gain on out-of-distribution CN/AD classification on the MR-RATE dataset, highlighting its effectiveness across both early-stage detection and cross-cohort generalization. For disease progression modeling, Prog-AG-JEPA consistently outperforms all baselines on future diagnosis and brain volume change prediction, demonstrating the value of anatomically grounded neurodegeneration representations for diagnosis and prognosis.

0.09

Longitudinal Bayesian Learning of Continuous Disease Position across the Alzheimer's Disease Continuum

cs.LG cs.AIq-bio.QM ·2026-08-19 ·Yingying Zhang, Kun Zhao, Guodong Liu, Qi Huang, Pengfei Gu +8 more
similarity 0.69
Abstract

Alzheimer's disease (AD) progresses as a continuous biological process, whereas most existing neuroimaging-based artificial intelligence methods remain limited to discrete diagnosis or clinical score prediction from cross-sectional imaging. In this work, we propose Disease Continuum Positioning (DCP), a longitudinal Bayesian Learning framework that continuously estimates disease severity from longitudinal diffusion tensor imaging (DTI). Specifically, DCP models disease severity as a low-dimensional probabilistic latent variable by jointly integrating longitudinal observations with weak clinical supervision, from which the proposed Disease Continuum Score (DCS) is derived to quantify an individual's position along the Alzheimer's disease continuum together with its associated uncertainty. Extensive experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort demonstrate that DCP consistently outperforms representative disease progression methods. More importantly, comprehensive validation analyses show that DCS accurately characterizes disease severity, exhibits strong clinical relevance, preserves longitudinal disease evolution, and predicts future disease conversion. These results suggest that DCS provides a quantitative imaging-derived representation for continuous assessment of Alzheimer's disease progression beyond conventional diagnostic labels and clinical scores.

0.19

GenFAR: A generalized representation of brain structure, derived from 49,246 multi-cohort MRIs via deep learning

cs.CV ·2026-08-12 ·Vishnu M. Bashyam, Guray Erus, Junhao Wen, Pratik Chaudhari, Randa Melhem +27 more
similarity 0.69
Abstract

Deep learning models for neuroimaging have largely been developed for individual tasks, limiting knowledge transfer across applications. Here we introduce GenFAR, a modular deep learning framework that learns general, clinically informed features from brain MRIs. We trained this modular architecture on 49,246 individuals across 11 cohorts, using 17 diverse classification and regression tasks spanning cognition, clinical, diagnosis, demographics, and biomarkers. This yields aggregated, focused feature sets that capture rich, clinically- and biologically-relevant brain representations. We developed a sequential learning approach where tasks progressively build on previously learned representations. Through an analysis of 5,000 task sequences, we identified an optimal sequence length of six tasks and introduced a Donor Score metric to quantify each task's contribution to downstream performance. This analysis revealed five consistently strong donor tasks (Age, AD/MCI, MMSE, Hypertension, Hyperlipidemia) that formed the base of our sequential model. We demonstrated the utility of our learned representation, in various tasks beyond those included in the training set, to serve as the foundation for specialized secondary predictors. We further showed that using the learned feature representation can substantially increase the sample efficiency of secondary deep learning training tasks and models, as well as improve their accuracy.

0.04

Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI

cs.CV cs.AI ·2026-09-14 ·Paul-Gabriel Nicolae, Irina Georgiana Mocanu
theory similarity 0.69
Abstract · 13 pages, 4 figures

Deep networks trained on structural MRI for Alzheimer's disease (AD) staging often reach reasonable accuracy while attending to anatomically irrelevant regions, and multimodal models that add clinical tables frequently rely on variables that were used to assign the diagnostic label in the first place. We study both issues with a deliberately lightweight slice-based encoder (ResNet18 with a one-layer Transformer over slices) on 1,075 baseline T1-weighted scans from ADNI-1. First, we use FastSurfer segmentations as an anatomical reference: YOLOv8 models trained on segmentation-derived labels localize Alzheimer-relevant structures with mAP_50 above 0.96, and a Grad-CAM comparison shows that the image-only classifier frequently attends to the skull, orbits and background. Second, we adapt a CLIP-style image - tabular contrastive framework and organize ADNIMERGE variables along a label-leakage spectrum. Fusion with cognitive scores yields 87.3% three-way accuracy, which we treat as a leakage-driven upper bound rather than an imaging result; fusion with regional volumes yields 73.0%. We observe that the choice of contrastive target changes what the image encoder learns: on MCI vs. CN, the image-only head reaches 52.4% when the encoder is aligned to cognitive scores and 73.8\% when aligned to volumes, although no tabular input is used at inference. Third, restricting the input to a per-subject crop of the medial temporal lobe raises image-only three-way accuracy from 58.7% to 65.1%. All results come from single runs on a small balanced test set, and we report confidence intervals and the protocol differences that prevent direct comparison with published numbers.

0.16