CS Paper Radar

Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration

Matteo Barbieri, Giammarco La Barbera, Juan Pablo De La Plata, Sabine Sarnacki, Isabelle Bloch, Pietro Gori

cs.CV cs.AI ·submitted 2026-09-14 14:47 UTC · v1
benchmark

Abstract

Multimodal image registration is a key component of many clinical workflows, yet it remains challenging because corresponding anatomical structures often exhibit substantially different image intensities across modalities. In this work, we present a comprehensive benchmark of intra-patient 3D multimodal deformable registration methods across three datasets covering different anatomical regions and difficulty levels, including both synthetic deformation recovery and real clinical scenarios. We evaluate classical optimization-based approaches and modern learning-based methods, including recent deep learning and foundation models, using complementary metrics: Average Dice similarity coefficient (DSC), average 95th-percentile Hausdorff distance (HD95), and a modality-independent structural similarity measure based on the MIND self-similarity context (MIND-SSC). Results show high variability across datasets, with learning-based methods demonstrating superior performance on large synthetic benchmarks, while only limited improvements are observed in real pelvic registration. A key finding of this study is the consistent disagreement between geometric metrics (DSC, HD95) and image-based similarity metrics (MIND-SSC), highlighting that improved overlap does not necessarily imply better global multimodal correspondence. Furthermore, anatomy-guided approaches achieve the highest overlap scores but exhibit degraded performance outside of segmented regions, revealing a trade-off between label-driven alignment and global structural coherence. Overall, our results indicate that no current method achieves robust performance across anatomies and modalities. We demonstrate that intra-patient 3D multimodal registration requires multi-criteria evaluation, including deformation-based metrics, and remains an open problem.

Comment
Preprint Under Review

Closest papers

Nearest neighbours by meaning across the whole index.

FlexiMIND: Combined Pretrained Vision Features and Local Structure for Training-Free Deformable Medical Image Registration

cs.CV physics.med-ph ·2026-10-25 ·Mojtaba Safari, Mingzhe Hu, Shansong Wang, Suyang Xi, Joshua Lorenz +4 more
code/data similarity 0.75
Abstract

Objective. Deformable image registration remains challenging because corresponding anatomy may exhibit different intensity relationships. Hand-crafted modality-robust descriptors such as Modality-Independent Neighborhood Descriptor with Self-Similarity Context (MIND-SSC) and features from pretrained vision encoders provide distinct representations for correspondence estimation, but how they should be combined remains unclear. We investigate whether tighter integration improves registration over a sequential, scale-separated design. Approach. We introduce FlexiMIND, a training-free and label-free two-stage framework in which frozen FlexiCT-2D encoder features establish contextual correspondence and MIND-SSC refines the deformation within ConvexAdam. Five alternative feature-coupling strategies were evaluated under a protocol across CT-CT (DIR-Lab 4DCT 4DCT, 10 cases), MR-MR (ACDC, 50 patients), and MR-CT (Learn2Reg Abdomen MR-CT, 8 pairs) registration. Performance was assessed using target registration error (TRE), Dice similarity coefficient (Dice), SDlogJ, and local folding. Main results. FlexiMIND achieved a TRE of 1.27 mm on DIR-Lab 4DCT, a DSC of 0.807 on ACDC, and an organ-averaged DSC of 0.825 on Abdomen MR-CT. Across 15 coupling-strategy dataset comparisons, only four outperformed the reference method, with a maximum absolute improvement of 0.0043 in Dice, whereas degradation reached 20.6 % in TRE and 37.9 % in Dice. Changes in registration accuracy were closely associated with changes in deformation-field regularity, and strategies that better preserved regularity generally remained closest to the reference performance. Landmark-level analysis showed that the pretrained encoder stage primarily reduced large correspondence errors rather than uniformly improving local precision. Code is available upon acceptance at https://github.com/mosaf/FlexiMIND.

0.24

Registration is a Representation Problem: Solving Whole-Body PET/CT Registration

cs.CV cs.LG ·2026-10-13 ·Neel Dey, Adrian V. Dalca
publishedbenchmarkcode/data similarity 0.69
Abstract · Technical report. First place winning solution for the MICCAI Learn2Reg 2026 PSMAReg challenge. Code and volumetric visualizations are avail…

Raw image intensities are a poor surrogate for similarity across images in correspondence problems. Further, while auxiliary segmentation labels can provide semantic identification and improve matching, they can assign only coarse identities to fine-grained anatomy and often cover only a subset of structures, particularly in whole-body settings. We present an approach that instead establishes correspondence using complementary representations of anatomy and demonstrate its utility for longitudinal whole-body PET/CT registration in the Learn2Reg 2026 PSMAReg Challenge. We use our 3D foundation model, anatomix, to derive fine-grained, per-voxel semantic representations that are consistent across volumes and further employ off-the-shelf segmentation models to obtain coarse anatomical identities. Together, these representations drive a GPU-accelerated iterative registration solver (FireANTs) in a hybrid multistage framework: the first stage estimates alignment from dense multi-channel features, after which a second refines the deformation using both anatomix- and segmentation-based losses. Lastly, because registration should not introduce non-biological tumor volume changes across time, we develop lesion-specific, component-wise volume preservation schemes. Without any task-specific training, our system placed first on the PSMAReg Challenge leaderboard. Code and interactive visualizations are available at https://www.neeldey.com/reps-for-reg.

0.31

Learning Where to Trust: Reliability-Guided Expert Routing and Velocity Gating for 3D Deformable Registration

cs.AI cs.CV ·2026-10-27 ·Chenyu Guo, Wenjie Dai, Yiqian Chang, Zhiyuan Zhang, Yunfan Wei +4 more
hot direction similarity 0.68
Abstract

Deformable image registration relies on accurate anatomical correspondences. However, explicitly coupling local matching reliability with spatial context selection and deformation update strength remains insufficiently explored in learning-based registration. This challenge is particularly pronounced in homogeneous, weakly textured, and severely deformed regions, where ambiguous matches may induce erroneous deformation updates. We propose ReliMorph, a reliability-routed symmetric framework for 3D deformable registration that couples correspondence assessment with spatial context selection and update strength.Local correlation distributions characterize correspondence ambiguity using shared anatomical features with modality-specific adaptation. These reliability cues guide spatial expert fusion and gate residual velocities during coarse-to-fine refinement. Updated inverse-inconsistency feedback guides routing and gating at finer pyramid levels. To our knowledge, ReliMorph is the first 3D deformable registration framework to jointly condition spatial expert routing and residual velocity gating on correspondence reliability. Extensive experiments demonstrate that ReliMorph consistently outperforms competing methods in anatomical overlap, highlighting its effectiveness across unimodal and multimodal registration tasks.

0.20

A thorough review on Deep Learning approaches to 2D-to-3D Registration in Medical Imaging

eess.IV cs.CVcs.LGphysics.med-ph ·2026-11-12 ·Andrea Borghesi, Julia van Beurden, Maarten van Alphen, Michiel van den Brekel, Luc Karssemakers +4 more
similarity 0.67
Abstract · 51 pages, 4 tables, 8 figures

2D-to-3D image registration is a key component for several medical imaging purposes such as intraoperative guidance, enabling the alignment between preoperative 3D volumes and real-time 2D images. While conventional optimisation-based methods often struggle with non-convex objectives, sensitivity to initialisation, and high computational demands, deep learning (DL) has emerged as a promising alternative, offering improved alignment robustness and the potential for real-time performance. This review provides a comprehensive analysis of DL frameworks developed to address the challenges inherent in 2D-to-3D registration. We first outline how core concepts of DL are adapted to the registration problem. We then examine recent methodological advancements from multiple perspectives, including pipeline design, architectural innovations, and domain adaptation strategies. Particular attention is given to quantitative metrics, highlighting how performance is defined and measured across studies. Across the 41 included studies, we find that direct regression is the most common pipeline (23 studies, against 18 iterative ones), that hybrid CNN-attention architectures are gaining ground, and that research remains heavily concentrated on X-ray-to-CT registration. We conclude by identifying the absence of per-case uncertainty estimation and the strong neglect of temporal or sequential modelling as the two methodological gaps most consequential for clinical readiness, and discuss state-space models as a way to add temporal memory within intraoperative latency budgets. Further opportunities lie in more principled domain adaptation and in the integration of foundation models for representation learning.

0.19

BINDER: A Latent Variable Model for Probabilistic Medical Image Registration

cs.CV ·2026-09-17 ·Stefano Cerri, Amirhossein Hassankhani, Yaël Balbastre, Koen Van Leemput
code/data similarity 0.66
Abstract

We propose a new probabilistic model for general-purpose medical image registration that builds upon the mutual information registration criterion. It centers around a spatial interpolation technique that assumes latent voxel-wise correspondences between the images being registered. By exploiting these latent variables, we derive dedicated optimization and MCMC sampling techniques that only involve closed-form iterative updates. When applied to nonlinear registration, an efficient demons-like optimization algorithm is obtained that shows robust out-of-the-box performance across a variety of monomodal and multimodal registration tasks. We also demonstrate a corresponding sampler that can quantify, for the first time, uncertainty in multimodal registration scenarios with very high-dimensional 3D deformations. Our code, which we call BINDER (Bayesian INference for DEformable Registration), is freely available at https://github.com/ste93ste/BINDER.

0.28

Architectural and Regularization Components in Deep Learning Medical Image Registration: Systematic Ablation Study

cs.CV cs.LG ·2026-08-23 ·Nabira Rashid
similarity 0.66
Abstract · 15 pages, 1 figure

Deep learning registration methods routinely stack two kinds of enhancement on a base network: architectural additions such as affine pre-alignment stages, and training-objective additions such as regularization losses. Papers tend to adopt both at once, so it is unclear which is doing the work. I ran a controlled ablation to separate them. Using the OASIS brain MRI dataset (394 training subjects, 20 test subjects), I trained four variants of the same registration pipeline: a baseline 3D U-Net with basic similarity losses, the same U-Net with a full regularization suite, an affine-plus-deformable architecture with basic losses, and the affine architecture with the full suite. I evaluated registration accuracy (MSE, NCC, SSIM), deformation quality (Jacobian determinant preservation, displacement statistics, an anatomical plausibility score), and computational cost. Regularization alone accounted for most of the gain: a 21.3% relative gain on the MSE-improvement metric (1.78% to 2.16%, P<.001) and a 21.8% relative gain in NCC improvement, while cutting maximum deformation from 53.1 to 0.51 units, a 99.0% reduction, at essentially no computational cost (-0.06% inference time). The combined model produced the largest accuracy gain, 25.8% (1.78% to 2.24%), and raised anatomical plausibility from 0.596 to 0.930, at a moderate +9.8% inference-time cost. Gradient correlation rose from 0.742 at baseline to 0.980 for the fully enhanced model. All enhanced variants reached sub-voxel accuracy under plausible deformation constraints. Regularization losses are the primary driver in this setting, delivering the accuracy gains and almost all of the deformation control for free at inference time, while the affine architecture adds a smaller complementary benefit at acceptable cost. The 99% reduction in unrealistic deformations addresses a known barrier to clinical deployment.

0.24

Unsupervised Multi-modal Medical Image Registration via Invertible Translation

cs.CV ·2026-11-17 ·Mengjie Guo, Jinming Duan
code/data similarity 0.66
Abstract

In medical imaging, the alignment of multi-modal images plays a critical role in providing comprehensive information for image-guided therapies. Despite its importance, multi-modal image registration poses significant challenges due to the complex and often unknown spatial relationships between different image modalities. To address this, we introduce a novel unsupervised translation-based multi-modal registration method, termed Invertible Neural Network-based Registration (INNReg). INNReg consists of an image-to-image translation network that converts multi-modal images into mono-modal counterparts and a registration network that uses the translated mono-modal images to align the multi-modal images. Specifically, to ensure the preservation of geometric consistency after image translation, we introduce an Invertible Neural Network (INN) that leverages a dynamic depthwise convolution-based local attention mechanism. Additionally, we design a novel barrier loss function based on Normalized Mutual Information to impose constraints on the registration network, which enhances the registration accuracy. The superior performance of INNReg is demonstrated through experiments on two public multi-modal medical image datasets, including MRI T1/T2 and MRI/CT pairs. The code is available at https://github.com/MeggieGuo/INNReg

0.41

Auditing Patient Privacy in Medical Generative Models: Scalable Memorization Detection with DeepSSIM++

cs.CV ·2026-09-03 ·Antonio Scardace, Francesco Guarnera, Sebastiano Battiato, Daniele Ravì
code/data similarity 0.66
Abstract

While deep generative models offer new opportunities for medical image synthesis and data sharing, their ability to memorize and reproduce training samples raises serious concerns about patient confidentiality. Detecting such memorization at scale remains challenging: traditional pixel-based metrics are sensitive to generation artifacts, whereas generic embedding-based metrics often lack the anatomical sensitivity required for medical data. To address this challenge, we introduce DeepSSIM++, a self-supervised similarity metric for scalable memorization auditing in medical generative models. By leveraging multi-scale feature aggregation and anatomy-preserving augmentations, DeepSSIM++ learns an embedding space where cosine similarity approximates the Structural Similarity Index (SSIM), eliminating the need for exact pixel-level registration. Compared with state-of-the-art baselines, DeepSSIM++ achieves an average Macro F1 improvement of 33 percentage points under ideal alignment and 46 percentage points under realistic spatial and intensity perturbations. Furthermore, it accelerates large-scale similarity computation by several orders of magnitude compared with analytical SSIM. By combining anatomical sensitivity and computational efficiency, DeepSSIM++ provides an open-source tool for scalable memorization auditing in medical generative AI. Code and data are publicly available at: https://github.com/brAIn-science/DeepSSIM.

0.30