CS Paper Radar

Brain-Behavior Modeling from Multimodal MRI using Opportunistic Pretraining and Common-Cohort Adaptation

Md. Tanvir Rahman, Taslima Ferdaus Shuva, Nabil Anan Orka, Asaduzzaman Khan, Mohammad Ali Moni

cs.CV ·submitted 2026-10-28 06:46 UTC · v1

Abstract

Incomplete imaging coverage limits sample sizes for multimodal cognitive prediction. We evaluated a two-stage opportunistic deep ensemble using baseline Adolescent Brain Cognitive Development Study data to predict fluid, crystallized, and total cognition. Separate neural models processed structural/diffusion, resting-state, and task-based MRI data with covariates. Models were pretrained on eligible modality cohorts (up to 9,686 participants), adapted to common training cohorts, and fused via Ridge regression (trimodal cohort: 4,216 participants). Five-fold family-aware cross-validation compared this approach against constituent and covariate-only models. Two-stage training increased mean test coefficient of determination (R^2) in 24 of 27 comparisons (median gain: 0.0081). Fusion increased mean R^2 by 0.0051-0.0333 over the strongest constituent and reduced mean absolute error by 0.0067-0.0976. Task-plus-resting-state and trimodal ensembles reduced absolute error across all targets (nominal p <= 0.0108). Task-plus-resting-state fusion achieved the highest mean R^2 for fluid (0.2222) and total cognition (0.3665), while structural-plus-resting-state led for crystallized cognition (0.3706), yielding R^2 gains of 0.0805, 0.1180, and 0.0890 over covariate-only references. Ablations showed expanded pretraining drove most improvements. Gradient-based attribution identified target-dependent input patterns. These findings support using partially observed cohorts for model development to achieve modest fusion gains. Pending external validation, this approach may support MRI-informed cognitive phenotyping.

Comment
39 pages, 4 figures. This work has been submitted to the Elsevier NeuroImage for possible publication

Closest papers

Nearest neighbours by meaning across the whole index.

MAC-Net: A Multi-Task Deep Learning Framework for Modeling Cognitive Function From Task-Based fMRI

cs.AI ·2026-09-27 ·Md. Tanvir Rahman, Nabil Anan Orka, Asaduzzaman Khan, Mohammad Ali Moni
similarity 0.71
Abstract · 13 pages, 4 figures, 2 supplementary tables. This work has been submitted to the IEEE Transactions on Neural Systems and Rehabilitation Engi…

Objective cognitive assessment from neural signals supports neurorehabilitation, but individual-level prediction from task-based fMRI (tfMRI) remains difficult because neural features coexist with substantial demographic and scanner-related variation. We present the Multi-task Activation and Contrast Network (MAC-Net), a covariate-aware deep learning framework for modeling individual cognitive function from regional tfMRI. By isolating tfMRI features into a dedicated neural pathway and restricting participant variables to a terminal late-fusion pathway, MAC-Net prevents dominant covariates from suppressing high-dimensional clinical representations during feature learning. Evaluating baseline data from 6,500 Adolescent Brain Cognitive Development Study participants under family-aware cross-validation, MAC-Net was benchmarked against linear models, random forests, and alternative deep architectures. The N-back plus Monetary Incentive Delay configuration achieved $R^{2}$ values of 0.174, 0.238, and 0.277 for fluid, crystallized, and total cognition, outperforming covariate-only baselines (0.178) and alternative deep models (0.217). N-back was the most informative paradigm, whereas incorporating the Stop Signal Task marginally degraded performance. Feature attributions via Integrated Gradients, DeepLIFT, and Input Gradient were highly concordant, localizing working-memory-related frontal, parietal, and cingulate regions. These findings demonstrate that covariate-aware multi-task modeling yields reproducible cognitive-function estimations, establishing a robust neural engineering framework for clinical translation.

0.15

BrainPFN: Amortized Brain-Behaviour Prediction via Prior-Data Fitted Networks

cs.AI q-bio.NCcs.LG ·2026-09-29 ·Carlo Alberto Barbano, Demian Wassermann
similarity 0.64
Abstract

The standard approach to predicting behavioral phenotypes from functional MRI is represented by Kernel Ridge Regression (KRR) applied on functional connectivity (FC) matrices. However, this approach is limited: separate models must be fit and tuned for every score and cohort, they do not transfer across datasets, and accuracy degrades sharply when only a handful of subjects are available, as is typical of clinical or cognitive studies. Despite an increasing interest in developing brain foundation models, none has consistently outperformed KRR on behaviour prediction; most reported gains come from easier targets such as age, sex, or diagnosis rather than continuous behavioural scores. We introduce BrainPFN, a Prior Data Fitted Network for brain behaviour regression. BrainPFN is meta-trained on a domain-specific prior built from ~7k real FC matrices, from which we sample one billion synthetic training samples. At inference, a labelled context is simply passed through the network in a single forward pass, with no per-score or per-cohort refitting required. Across four independent cohorts, BrainPFN beats KRR on small sample prediction, and leads recent foundation models pre-trained at four to nine times larger scale. Our meta-training pool, aggregated from open repositories, is a fraction of the size of the corpora behind competing models, most of which sit behind restrictive data use agreements such as UK Biobank. We release this pool publicly alongside code and checkpoints, showing that a modest, openly available pool can match or beat models trained at Biobank scale.

0.35

CoFlu-Brain: An fMRI Connectivity Foundation Model with Cofluctuation-Based Pretraining

cs.NE ·2026-10-11 ·Ziteng Sui, Junfeng Xia, Wenhao Ye, Mo Wang, Quanying Liu
code/data similarity 0.63
Abstract

Functional magnetic resonance imaging (fMRI) commonly characterizes large-scale brain organization through functional connectivity (FC). However, an observed FC matrix is not a fixed brain graph but a statistical estimate determined by the finite BOLD frames used to compute it. Conventional FC-based approaches typically represent each scan with a single Pearson-correlation matrix, leaving frame-resolved variations in regional interactions unexploited. To address this limitation, we introduce CoFlu-Brain, which organizes framewise cofluctuation observations into multiple retained and dropped views, each summarized as an FC matrix. The model learns connectivity structure reproducible across retained views and predicts the representations of the corresponding dropped views, while adopting SIGReg from LeJEPA to prevent representation collapse. We pretrained CoFlu-Brain on 13,444 subjects from eight dataset groups and evaluated its representations via frozen-encoder linear probing on seven classification and two age-prediction tasks across seven datasets. CoFlu-Brain showed strong transfer across demographic, developmental, and clinical prediction tasks, outperforming the evaluated released models on most tasks. Random frame partitioning improved aggregate transfer, while the complete objective achieved the strongest overall performance. Code is available.

0.20

Leakage-Controlled Multimodal Learning for Diagnosis and Progression Prediction in Alzheimer's Disease Research

cs.LG ·2026-10-07 ·Akeem Temitope Otapo, Ghazaleh Khodabandelou, Zuheng Ming, Alice Othmani
similarity 0.63
Abstract

Alzheimer's disease prediction involves irregular visits, heterogeneous measurements and incomplete modalities. This study presents a multimodal multitask framework combining an adapted SFCN MRI encoder, four causal clinical Transformers, shared fusion and task-specific ODE-GRU dynamics. Fine-tuning and LoRA adapt the final two MRI blocks. Task-DRO balances task losses, while Group-CVaR targets cohort and comorbidity strata. Branch-specific input controls, subject-grouped partitions and empirical causality checks support longitudinal evaluation. Across 2,649 subjects and 17,317 visits from ADNI, OASIS-2 and MIRIAD, internal validation yields diagnosis, stage-1 progression and first-stage-1-visit progression AUROCs of 0.935 +/- 0.002, 0.884 +/- 0.003 and 0.870 +/- 0.005, respectively (mean +/- SD across three seeds). Corresponding hybrid AUROCs are 0.951, 0.909 and 0.896. Next-visit MMSE mean absolute error (MAE) is 1.61 points; worst-stratum diagnosis AUROC is 0.827 +/- 0.008. Sampled ADNI explanations identify task-specific input dependence. OASIS-3 external validation yields network and hybrid diagnosis AUROCs of 0.763 and 0.767, hybrid next-visit progression AUROC of 0.764, diagnosis calibration error decreasing from 0.197 to 0.052, and next-visit MMSE MAE of 0.86. Seed-42 paired ablations of six components yield pooled diagnosis and progression AUROC differences between -0.004 and +0.004; removing clinical encoder inputs lowers diagnosis AUROC by 0.272. The framework integrates longitudinal prediction, missing-modality handling, auxiliary comorbidity modelling and subgroup evaluation within a common pipeline.

0.11

A generalizable structural brain MRI foundation model via dual-priority federated pretraining

cs.CV ·2026-09-23 ·Zhen Yu, Yang Liu, Jianfeng Feng, Xiahai Zhuang, Qingchao Chen
similarity 0.61
Abstract

Foundation models hold promise for generalizable analysis of structural brain magnetic resonance imaging (MRI) scans across diverse populations and imaging conditions. However, existing models typically rely on centralized pretraining using data pooled across cohorts, which is constrained by privacy and data-governance requirements. Moreover, larger cohorts can dominate optimization, underutilizing anatomical information from smaller cohorts and limiting generalization. Here we present BrainFedFM, a structural brain MRI foundation model federatively pretrained on 164,707 three-dimensional scans drawn from diverse real-world data distributions and organized across 42 federated sites. BrainFedFM uses dual-priority federated pretraining, in which spatial-priority masking emphasizes informative anatomical regions during local training, while site-priority aggregation adaptively weights contributions during global aggregation. Across 20 downstream datasets spanning 17 classification, regression and segmentation tasks, BrainFedFM achieved the state-of-the-art performance across seven foundation models, including four centralized models, while showing consistent advantages in classification and regression tasks and strong robustness under underrepresented populations and challenging imaging conditions. These findings demonstrate the generalizability of BrainFedFM and highlight federated pretraining as a practical strategy for developing neuroimaging foundation models from distributed data without pooling raw images.

0.25

When Model Complexity Matters in Cross-Subject Naturalistic fMRI Prediction

cs.AI q-bio.NC ·2026-11-02 ·Junhao Zhao, Jacob Kang, Colin Kurniawan, David Simberg, Nan Xu
similarity 0.61
Abstract · 43 pages

Increasingly complex neural models predict blood-oxygen-level-dependent (BOLD) responses to naturalistic stimuli, but the predictive value of this added complexity remains unclear. We ask when pretrained stimulus representations help and how much downstream complexity is needed, with and without recent measured BOLD history. We compare linear diagnostic references and nonlinear predictors across HCP movie fMRI (154 subjects), BOLD Moments (10 subjects), and Algonauts 2025 (4 subjects). Under a shared offline protocol, models are fitted on training subjects and evaluated on held-out subjects viewing largely familiar stimuli, without refitting. Pretrained visual features generally improve prediction over raw pixels; for linear autoregression, this advantage becomes more pronounced over longer rollouts (recursive predictions without additional measured BOLD). Measured BOLD history provides substantial gains that generally diminish {as rollouts lengthen}. Nonlinear stimulus-only predictors can improve on linear readouts, whereas linear autoregression provides a strong history-conditioned baseline. Controls examine context length, regularization, fitting procedures, and model capacity. Regional response consistency across subjects is associated with held-out prediction accuracy, while less consistent regions tend to gain more from measured history. In paired simulations with fixed linear neural dynamics, slower hemodynamic responses generally sustain high temporal prediction correlations over longer rollouts, illustrating how the observation process can contribute to predictive persistence. Together, these findings support evaluating stimulus representations and downstream complexity separately, using history-conditioned linear prediction as a diagnostic reference under matched inputs and measurement availability.

0.23

Explainable Deep Learning of Resting-State Functional Connectomes Reveals Network Biomarkers of Adolescent Intelligence

cs.AI q-bio.NC ·2026-09-27 ·Md. Tanvir Rahman, Nabil Anan Orka, Asaduzzaman Khan, Mohammad Ali Moni
similarity 0.61
Abstract · 12 pages, 2 figures. This work has been submitted to the IEEE Journal of Biomedical and Health Informatics for possible publication

Mapping resting-state brain organization to individual differences in cognitive ability remains a major challenge in population neuroinformatics. Although deep learning enables flexible modeling of brain connectivity, limited interpretability restricts its scientific and clinical utility. To address this objective, we developed an explainable deep learning framework based on sparse projected residual networks to predict fluid, crystallized, and total intelligence from resting-state functional magnetic resonance imaging in 5,285 participants from the Adolescent Brain Cognitive Development study. We incorporated three complementary explainability methods (Integrated Gradients, Gradient Shapley Additive Explanations, and Occlusion) to interpret model behavior. The framework outperformed existing approaches, achieving Pearson correlations of 0.44, 0.58, and 0.56 for fluid, crystallized, and total intelligence, respectively, corresponding to predictive improvements of 6 to 9 percent. All three explainability methods produced near-identical feature rankings (pairwise rank correlations greater than 0.99). Consensus maps revealed a dual-layered functional architecture where primary predictive hubs localized within canonical systems, while the strongest global predictive pathways frequently bypassed these hubs through distributed, long-range relay connections. These findings suggest that intelligence emerges from the interaction between localized computational hubs and distributed communication pathways. Ultimately, these normative network architectures provide clinical reference maps to detect individual deviations, supporting earlier diagnosis, cognitive subtype stratification, and treatment monitoring in atypical neurodevelopment.

0.32

BrainTaskonomy: Learning How to Pretrain and What to Transfer in fMRI Foundation Models

cs.CV q-bio.NC ·2026-09-09 ·Junfeng Xia, Wenhao Ye, Junxiang Zhang, Jiayu Zuo, Mo Wang +1 more
similarity 0.61
Abstract

fMRI foundation models increasingly aggregate heterogeneous data across brain states, cohorts, and acquisition settings, yet pretraining domains are commonly treated as a flat mixture and downstream tasks are adapted independently. We study whether measured learning relations can organize both stages without modifying the backbone. During pretraining, a lightweight Brain-DiT proxy estimates difficulty and directed facilitation across ten fMRI domains, yielding a priority-guided cumulative domain curriculum combined with high-to-low-noise timestep scheduling and joint consolidation. During adaptation, controlled first- and higher-order transfer across fifteen tasks constructs a directed taskonomy, from which budgeted integer programming (BIP) selects directly supervised source tasks and target-specific routes. The joint priority-domain and high-to-low-timestep curriculum reduces v-NMSE, PSD-NMSE, and FC-MSE by 6.5%, 16.3%, and 10.5%, respectively, relative to uniform sampling over both dimensions, and shows strong downstream performance across six in- and out-of-domain tasks. The taskonomy reveals asymmetric, target-dependent transfer, while exploratory sealed-test evaluation shows larger descriptive gains for BIP policies when higher-order route spaces are available than for matched random controls. Together, these findings support organizing fMRI pretraining and adaptation by measured learning relations rather than treating domains and tasks as independent flat sets.

0.55