CS Paper Radar

CNsEMD: An Expert-Annotated Multi-Field-Strength MRI Dataset and a Hyperspherical Manifold Network for Multimodal Cranial Nerve Parcellation

Lei Xie, Junxiong Huang, Guoqiang Xie, Jiawei Zhang, Jiahao Huang, Tianling Lyu, Ye Wu, Mingchu Li, Shoujun Yu, Shanshan Wang, Qingrun Zeng, Yuanjing Feng

cs.CV ·submitted 2026-09-07 05:28 UTC · v1
benchmarkdataset

Abstract

Cranial nerves (CNs) play essential roles in sensory, motor, and autonomic functions. Accurate CN parcellation from multimodal magnetic resonance imaging (MRI) is crucial for neuroanatomical analysis and neurosurgical planning. However, accurate CN parcellation remains extremely challenging because CNs are very small, exhibit low image contrast, and have slender tubular morphologies and complex anatomical trajectories. Moreover, the lack of publicly available, expert-annotated datasets has impeded the development and fair benchmarking of learning-based CN analysis methods. In this work, we introduce CNsEMD, an expert-annotated multimodal dataset for CN parcellation. It comprises data from 202 subjects acquired on 3T, 5T, and 7T MRI scanners. We further propose the projective hyperspherical manifold network (PHM-Net), which learns cross-modal representations by capturing angular relationships in a shared hyperspherical embedding space. Rather than performing multimodal fusion in Euclidean space, the proposed Hyperspherical cross-modal interaction (HCI) module enables bidirectional feature exchange between T1-weighted (T1w) and direction-encoded color (DEC) representations on a unit hypersphere. The Magnitude-preserving projective hyperspherical orientation representation (PHOR) captures the axial nature of DEC orientations while preserving diffusion magnitude. The hyperspherical prototype segmentation head (HPSH) further extends angular similarity to voxel-wise classification using normalized voxel embeddings and learnable class prototypes. Extensive experimental results on the CNsEMD dataset demonstrate the effectiveness of our PHM-Net against state-of-the-art methods. CNsEMD establishes a reproducible benchmark for multimodal CN imaging, while PHM-Net provides a geometry-consistent solution for CN parcellation across diverse MRI acquisitions.

Closest papers

Nearest neighbours by meaning across the whole index.

Mr3D-VL: A generalist vision language foundation model for Multiparametric 3D Magnetic Resonance Imaging

cs.CV cs.AI ·2026-08-13 ·Zhi Qiao, Xintong Wu, Yichu He, Feng Shi
similarity 0.68
Abstract

Multi-parametric magnetic resonance imaging (mpMRI) is a cornerstone for brain tumor diagnosis and treatment, yet current AI models face critical limitations: their lack of natural language interaction and interpretability impedes spatial information integration and cross-modal reasoning required clinically. Key challenges arise from significant physical meaning differences across modalities, spatial misalignment due to scan intervals, and the need for complex multi-feature interpretation in tasks like glioma grading. While visual-language models (VLMs) show promise in cross-modal understanding, existing methods focus mainly on 2D image modeling, neglecting direct perception of 3D volumetric space. Although 3D VLMs have been proposed for report generation and feature alignment in 3D CT imaging, mpMRI applications demand collaborative inference across multiple imaging modalities-a requirement unmet by current solutions. To address this, we introduce Mr3D-VL, a dedicated visual-language foundation model for multi-parametric 3D MRI. With 4 billion parameters, it employs an unsupervised pre-trained shared 3D encoder and 4D rotational positional embedding for dual modality-spatial integration. Its cross-modal projection layer uses a multi-resolution feature implantation strategy to enhance feature perception across resolutions. Experimental results show significant improvements over existing 4B/7B/30B domain-specific and general-purpose models in text generation tasks, achieving a BERTScore of 0.856 for report generation, with question-answering accuracy at 0.713 and multiple-choice accuracy at 0.912.

0.04

Learning Subject-Specific Anatomical Representations via Manifold Expansion: Application to Accelerated Multi-Contrast MRI

cs.CV cs.IR ·2026-10-02 ·Ruimin Feng, Wanyu Bian, Albert Jang, Zachary Stewart, Fang Liu
similarity 0.66
Abstract

Clinical MRI routinely acquires multiple contrast-weighted images of the same anatomy for complementary tissue characterization. However, current accelerated MRI methods typically reconstruct each contrast independently, without fully exploiting shared anatomical information. This work aims to learn anatomical representations invariant to contrast-dependent appearance for reconstruction of accelerated multi-contrast MRI. We propose MAX (MAnifold eXpansion), a subject-specific framework that learns anatomical representations from a single fully sampled reference contrast. To address the under-constrained separation of shared anatomy and contrast-dependent components from a single image, MAX expands the multi-contrast manifold using anatomy-preserving intensity augmentations. A disentangled implicit neural representation models augmented samples using shared spatial coordinates for anatomy and spatially invariant coordinates for contrast appearance. The learned anatomical representation is then fixed, with the contrast representation adapted to the undersampled target data, followed by unrolled refinement. Theoretical analyses further provide insight into the disentangled representation learning and explain how the learned anatomical representation improves the target contrast reconstruction. At R = 8 for brain MRI and R = 6 for knee MRI, MAX achieves the highest mean PSNR and SSIM across all tasks, improving PSNR by more than 1 dB over the strongest baseline for both brain contrasts. MAX more faithfully recovers subtle anatomical and pathological structures and remains robust to inter-contrast motion, structural heterogeneity between reference and target contrasts, and measurement noise. Therefore, MAX provides a general strategy for leveraging high-quality reference scans in accelerated MRI and has the potential to be extended to other reference-assisted MRI inverse problems.

0.12

Enabling Vision and Cross-Modal Learning for Multimodal Stroke Recurrence Prediction: An Interpretable Two-Step Framework

cs.CV cs.AI ·2026-09-10 ·Christian Gapp, Elias Tappeiner, Martin Welk, Karl Fritscher, Stephanie Mangesius +6 more
publishedcode/data similarity 0.64
Abstract · ML-CDS 2026: Multimodal Learning and Fusion Across Scales for Clinical Decision Support, MICCAI 2026, Strasbourg, France

Multimodal stroke recurrence prediction requires effective integration of heterogeneous clinical and imaging data, yet modality imbalance often causes models to over-rely on dominant modalities and underutilize complementary information. While self-supervised pretraining and selective parameter freezing are commonly employed to improve representation learning and fine-tuning stability, their effect on modality contributions and cross-modal behavior in multimodal medical models remains largely unexplored. In this work, we investigate whether image pretraining on 3D CTA scans reduces modality imbalance and improves cross-modal integration for stroke recurrence prediction, a clinically critical task we recently addressed. To this end, two multimodal neural networks are pretrained in a self-supervised manner and subsequently fine-tuned using two distinct freezing strategies. Their performance and modality utilization are compared against both the baseline model from our previous work and models trained entirely from scratch in this study. Our results demonstrate that self-supervised pretraining enables more effective utilization of the multimodal image-tabular dataset, outperforming both the prior baseline and all non-pretrained models. Notably, the best-performing Vision Transformer based neural network successfully overcomes unimodal collapse. Synergy analysis reveals significant interactions between vision and both gender and CHD, suggesting clinically relevant patterns for stroke recurrence. Overall, our findings demonstrate that self-supervised pretraining and strategic fine-tuning support more balanced modality utilization and enable meaningful cross-modal interactions. Code is publicly available at https://github.com/ChristianGappGit/SSL_Pretraining.

0.48

VoxForge: A Unified Foundation Model for Dense 3D Features Across MRI, CT and PET

cs.CV cs.LG ·2026-11-19 ·Rushil Anirudh, Jayaraman J. Thiagarajan
similarity 0.64
Abstract · 34 pages (8-page main paper, references, and 23-page supplementary material), 7 figures. Rushil Anirudh and Jayaraman J. Thiagarajan contrib…

A unified 3D medical foundation model promises reusable, frozen features that serve a wide variety of tasks such as segmentation, classification and registration on MRI, CT and PET as well as specialist models do. Generalist models still need separate backbones for localized and global tasks, specialist models are confined to a single modality, with the concern that spanning modalities under-emphasizes fine anatomy, and both typically rely on end-to-end fine-tuning to be competitive. We present VoxForge, a single encoder that integrates recent advances in 3D architecture, self-distillation and cross-modal curricula to learn transferable features from permissively licensed MRI and CT volumes. We evaluate it strictly frozen, the test a reusable foundation model must pass, against state-of-the-art generalist and specialist models on segmentation, classification and registration across MRI and CT, and on PET, which it never sees. VoxForge has the best or tied-best average rank in every task group, serves localized and global tasks across modalities at the level of state-of-the-art specialists, and is extremely label-efficient, matching top-performing models on TotalSegmentator CT with a tenth of the labels.

0.18

Cross-Dimension and Cross-Modality Relational Tensor Representation Learning for Multimodal Health Data Modeling

cs.LG ·2026-10-17 ·Jiawei Guo, Meng Zhao
similarity 0.64
Abstract · 30 pages

Multimodal health data integrate information from heterogeneous clinical sources, creating substantial opportunities for data-driven decision support. Realizing these opportunities, however, requires effectively modeling complex dependencies that arise both within individual modalities and across modalities. This paper proposes Relational Tensor Representation (RTR), a unified framework that jointly learns dimension-specific tensor representations and cross-dimensional and cross-modality relationships. RTR introduces learnable relational bridge matrices that maintain relational information while preserving the structural identities of tensor dimensions and modalities. A structured fusion mechanism further integrates factor-level and relationship-level information into compact representations for downstream prediction. Experiments on simulated data and two multimodal neurophysiological datasets, covering emotion recognition and brain activity classification, demonstrate that RTR consistently outperforms representative tensor learning, multimodal fusion, and Transformer-based baselines. These results highlight the value of explicitly modeling cross-dimensional and cross-modality relationships for learning informative representations from heterogeneous multimodal tensor data. The source code is planned for public release to support reproducibility and future research.

0.14

Structural-Functional Brain Connectivity Generation via Multimodal Hypergraph-based Flow Matching

cs.LG ·2026-10-02 ·Chyong Yi Poh, Hwa Hui Tew, Junn Yong Loo, Raphaël C. -W. Phan, Fuad Noman +2 more
similarity 0.63
Abstract · 10 pages, 3 figures, 5 tables

Structural connectivity (SC) and functional connectivity (FC) provide complementary information on interactions between brain regions and are widely used in neuroimaging studies of neuropsychiatric disorders. Generative modelling can alleviate the scarcity of large-scale paired SC-FC data, but existing approaches typically use pairwise graphs that capture only dyadic interactions and often generate SC and FC independently, limiting preservation of higher-order structure-function relationships. We propose a Multimodal Hypergraph Flow Matching (MHG-FM) framework for joint SC-FC connectivity generation and cross-modal translation. MHG-FM constructs modality-specific hypergraphs, learns higher-order representations with Hypergraph Neural Network (HGNN) encoders, and performs bidirectional cross-modal fusion using Dual Cross-Attention (DCA). A variational autoencoder maps the fused representations to a compact latent space, where conditional flow matching enables connectivity synthesis and multimodal translation via latent transport. Experiments on the Human Connectome Project Young Adult (HCP-YA) dataset show that MHG-FM outperforms several state-of-the-art baselines in reconstruction quality, topology preservation, distributional similarity, and SC-FC coupling, while achieving approximately 8x faster sampling than a matched diffusion backbone.

0.21

Multi-Resolution Feature Fusion U-Net for Magnetic Resonance Imaging Segmentation

cs.CV cs.LG ·2026-09-24 ·Eirini Cholopoulou, Dimitrios E. Diamantis, Dimitris K. Iakovidis
publishedbenchmark similarity 0.63
Abstract

The segmentation of anatomical structures in medical images and particularly in MRI scans, is essential for clinical diagnosis and monitoring disease progression. While Deep Learning (DL) architectures, such as U-Net and its extensions are very effective in medical image segmentation tasks, they often struggle with preserving fine-grained details and global contextual information. This is especially challenging for MRI data segmentation, where anatomical structures are characterized by irregular boundaries and variations in shape, contrast, and scale. To address this challenge, we propose a novel DL architecture for MRI segmentation across different anatomical structures. Specifically, the architecture introduces a module, named Multi-Resolution Feature Fusion (MRFF), that can be easily integrated into any U-Net-like architecture. The MRFF is integrated in all levels of an encode-decoder structure, along with attention mechanisms and skip connections to extract features at multiple resolutions, enabling the model to capture both fine-grained details and global contextual information. We evaluate the MRFFU-Net on two publicly available benchmark MRI datasets of different anatomical targets; one for Cerebrospinal Fluid (CSF) segmentation in spinal MR scans, and one for left atrium cardiac segmentation, from the Medical Segmentation Decathlon (MSD) challenge. Experimental results indicate that MRFFU-Net outperforms state-of-the-art models across multiple evaluation metrics, demonstrating its effectiveness in MRI segmentation.

0.45

dMRI-F: A Foundation Model for Generalizable Diffusion MRI Analysis from Sparse Acquisitions

cs.CV ·2026-10-13 ·Xi Zhu, Yijie Li, Xiaofan Wang, Wei Zhang, Huafu Chen +2 more
similarity 0.63
Abstract

Diffusion MRI (dMRI) probes water diffusion within the brain, providing unique information about tissue microstructure and white matter connectivity. High-quality dMRI enables advanced and accurate analyses but demands dense acquisitions, resulting in long scan times and hardware constraints that are often infeasible. Further, dMRI requires dedicated computational methods to support diverse analysis tasks (e.g., parcellation, registration, and synthesis) for both neuroscientific and clinical applications. Here, we present dMRI-F, a foundation model that unifies dMRI data enhancement and downstream tasks across diverse acquisitions and populations, while requiring only minimal acquisition input. dMRI-F is pre-trained with a q-space masked reconstruction task: predicting densely sampled multi-shell data from only six volumes. It learns generalizable latent feature representations of dMRI data, which serve as a core foundation for adapting to diverse downstream tasks, including brain parcellation, brain registration, and cross-modal synthesis. Across nine independently acquired datasets, we demonstrate that dMRI-F outperforms state-of-the-art methods with robust generalization across acquisitions, populations, and tasks. Our results demonstrate dMRI-F's versatility as a general-purpose framework that enables diverse dMRI applications from low-cost, sparse acquisitions.

0.15