CS Paper Radar

CNVAE: Anatomically Correct Segmentation via Conditional Generative Modeling

Freddy Jiang, Loic Le Folgoc, Elsa Angelini

eess.IV ·submitted 2026-11-04 22:12 UTC · v1

Abstract

Supervised deep learning segmentation models, although unparalleled in performance, are not immune to anatomical mistakes. We develop a segmentation-by-generation paradigm extending the Nouveau VAE (NVAE) generative framework for anatomically correct cardiac segmentation. In particular, we propose a novel conditional NVAE model (CNVAE) for conditional shape generation, and explore its capabilities for image segmentation in two scenarios: (1) direct image-to-mask CNVAE for one-step segmentation; (2) mask-to-mask CNVAE for refinement of segmentation masks produced by third-party segmentation models. Both models are lightweight and return consistently anatomically valid segmentation masks (> 98% success rate, an improvement of 15% above state-of-the-art segmentation models) on the cardiac segmentation/editing tasks of interest, without loss of accuracy in segmented contours.

Closest papers

Nearest neighbours by meaning across the whole index.

Improving Cross-Site Whole-Heart Segmentation

eess.IV cs.CV ·2026-08-25 ·Tanish Mudaliar, Justin Li, Daniel Lin, Julianna Vo, Kaitao Liao +2 more
publishedcode/data similarity 0.70
Abstract · 12 pages, 2 figures. Accepted to the MICCAI 2026 for the CARE Whole Heart Segmentation Challenge proceedings

Whole-heart segmentation from CT and MRI is essential for quantitative cardiac image analysis, but remains challenging under multi-center and multi-modality distribution shift. In the CARE whole-heart segmentation task, models must generalize from limited labeled sites to unseen acquisition distributions, where variation in spacing, intensity, reconstruction texture, and anatomy can degrade out-of-distribution performance. We propose a modality-routed 3D cardiac segmentation pipeline that combines TotalSegmentator-initialized nnU-Netv2 models with site-characterized, label-preserving appearance augmentation. We first characterize the available sites using measurable image properties and use this analysis to motivate candidate data-space generalization routes. The final retained recipe applies Bias Field + Bezier appearance augmentation, combining smooth spatial intensity perturbation with nonlinear intensity remapping, followed by lightweight class-wise largest-connected-component cleanup. On the primary held-out-site validation splits, the final configuration improves CT mean Dice from 0.8350 to 0.9135 and MRI mean Dice from 0.7695 to 0.7830, while also reducing HD95. These results suggest that site-motivated appearance augmentation is a practical strategy for improving cross-site robustness in limited-data whole-heart segmentation. Our code can be found in https://github.com/Purdue-M2/Improving-Cross-Site-Whole-Heart-Segmentation

0.51

SV-Cine: Diagnosis-Conditioned Segmentation of Single Ventricle Physiology via Generative Data Augmentation

cs.CV physics.med-ph ·2026-09-11 ·Lila Cunge
similarity 0.69
Abstract · arXiv admin comment: This version has been removed by arXiv administrators as the submitter did not have the rights to agree to the license …

Single Ventricle Physiology (SVP) is a rare subtype of congenital heart disease characterized by the presence of a single functional cardiac ventricle with atypical anatomic configurations that challenge conventional image segmentation approaches. The scarcity of clinical data and the morphological diversity across SVP subtypes make the development of robust segmentation methods particularly difficult. To address these limitations, we propose a cardiac MRI segmentation framework focused on ventricular chambers and myocardium segmentation tailored for SVP. First, we introduce a data augmentation pipeline that generates synthetic 3D cardiac meshes using SDF4CHD and corresponding synthetic cardiac MRI through generative modeling. Second, we introduce SV-Cine, a diagnosis-conditioned adaptation of the foundation model CineMA that incorporates patient-level diagnostic information through Feature-wise Linear Modulation layers, enabling diagnosis-aware feature adaptation during segmentation. We evaluated the framework on an internal cohort with varying SVP subtypes. SV-Cine achieved median Dice scores of 0.89 (IQR: 0.80--0.91) for the left ventricle and 0.72 (IQR: 0.54--0.84) for the right ventricle, outperforming the strongest baseline, nnU-Net, by 0.39 Dice points on right ventricle segmentation. It also yields a median ejection fraction error of 5.55 percentage points (IQR: 3.41--7.69) for the dominant ventricle. Compared with the internal cohort, LV and myocardium segmentation performance was lower for the external cohort; whereas RV Dice scores were comparable for both cohorts. Our findings suggest that a pretrained foundation model can be adapted for highly specialized downstream tasks through usage of diagnosis priors while leveraging anatomic knowledge learned from large-scale MRI datasets during pretraining.

0.12

InstEditSeg: Instruction-Driven Image Editing for Polyp and Skin Lesion Segmentation

cs.CV cs.AI ·2026-09-02 ·Ziquan Liu, Zhewei Zhu, Xuyang Shi
code/data similarity 0.67
Abstract · 21 pages, 11 figures

Accurate segmentation of polyps and skin lesions is pivotal for clinical diagnosis, yet existing methods struggle with low contrast, ambiguous boundaries, and cross-domain distribution discrepancies. Discriminative networks and most diffusion-based segmentation approaches predict standalone binary masks, leaving the visual priors of large-scale pretrained generative models largely unexploited. We propose InstEditSeg, a unified generative framework that reformulates medical segmentation as an instruction-driven image editing problem. Instead of emitting a mask, the model renders a color-coded overlay on the original image, conditioned on a textual instruction, so that the edited output aligns with the natural image distribution learned by latent diffusion models and mitigates the domain gap between natural and medical imagery. To recover fine anatomical structures, we introduce DINOv3 as an auxiliary visual encoder and a DINO Feature Guidance Block that builds a multi-scale feature pyramid. The pyramid is fused into the diffusion U-Net by channel concatenation and zero-initialized convolution so that hierarchical discriminative priors can be injected without perturbing the pretrained weights. A dual-branch classifier-free guidance strategy requiring only two forward passes per denoising step reduces inference cost. On polyp and skin lesion benchmarks the framework achieves accuracy competitive with strong discriminative baselines, and it further demonstrates concrete advantages of the generative formulation: notably better cross-domain generalization on unseen data, more complete multi-lesion segmentation, instruction-conditioned task control, and sampling flexibility. We also analyze the strengths and limitations of the paradigm, including its color sensitivity and unsupported attribute-conditioned selection. Code is available at: https://github.com/wincharm001/InstEditSeg.

0.34

DART: Scaling a Shape Representation Across Anatomical Classes and Imaging Modalities

cs.CV ·2026-10-20 ·Koushik Swarna, Pedro R. A. S. Bassi, Lam Ngo, Wenxuan Li, Jiancheng Yang +2 more
code/data similarity 0.65
Abstract

Current state-of-the-art segmentation models still produce anatomically implausible shapes, such as broken vessels and fragmented glands, on the structures that are hardest to segment. Learned shape representations capture what structures look like and can help correct these errors, but most cover a fixed set of anatomical classes. Extending this coverage across classes and imaging modalities is costly because it often requires new voxel-level annotations. We propose Diffusion for Anatomical Reshaping with Text (DART), which learns a shared shape representation from segmentation probabilities. It uses anatomical descriptions to correct these probabilities without receiving medical images. Across 13 public benchmarks, absolute Dice gains reach 31% for individual gallbladder cases in CT and 10% for individual prostate cases in MRI. Using existing segmentation predictions, we scale a single shape representation from 50 to 377 anatomical classes without new voxel-level annotations for correction training. The shared representation also corrects four lesion classes absent from training and transfers from CT to MRI without retraining. Corrected labels also improve segmentation models, raising one model's Dice from 57.1% to 67.5%. The same pipeline supports annotations for 45,000 CT scans. These results suggest that sharing shape information within and across classes can reduce the need for new voxel-level annotations when extending segmentation to more anatomical classes and imaging modalities. The model is available at https://huggingface.co/Koushik45/DART-Seg

0.26

Flow-based conditional cardiac anatomy generation for virtual cohorts

cs.LG cs.CVq-bio.QMq-bio.TO ·2026-08-10 ·Konstantinos Kevopoulos, Beatrice Moscoloni, Benjamin Alheit, Cameron Beeche, Julio A. Chirinos +2 more
similarity 0.65
Abstract

Cardiac digital twin research is moving from subject-specific anatomical replicas toward virtual cohorts that represent clinically relevant population subgroups. Yet access to representative imaging-derived anatomy datasets remains limited by cohort size, subgroup sparsity, and data-sharing constraints. Conditional generative models could help address this gap, but virtual cohorts are useful only if they preserve realistic, metadata-dependent anatomical variability. Existing cardiac anatomy generators largely rely on conditional variational autoencoders (cVAEs), which couple representation learning and metadata conditioning through a shared regularized latent prior. We introduce CAN-FLOW, a two-step Conditional ANatomy generation framework based on normalizing FLOWs that first learns geometry-only latent representations of diffeomorphic cardiac shape momenta and then models their sex-, age-, and body-mass-index-dependent distribution with a conditional normalizing flow. We trained CAN-FLOW on 2,208 healthy UK Biobank subjects and compared it with cVAEs across regularization strengths. CAN-FLOW generated plausible stochastic biventricular anatomies that better reproduced clinical phenotype distributions, metadata-dependent trends, subgroup variability, point-cloud coverage, and high-dimensional shape variability. Together, these results establish CAN-FLOW as a shareable framework for generating realistic, stochastically varying, metadata-conditioned biventricular anatomies for virtual cohort construction and in silico clinical trial workflows.

0.04

Flow Matching Meets 3D Curvilinear Structure Segmentation in Medical Imaging

cs.CV cs.LGq-bio.QM ·2026-08-20 ·Sidi Mohamed Sid'El Moctar, Nicolas Vitry, Hélène Bouvrais
published similarity 0.65
Abstract · International Workshop on Machine Learning in Medical Imaging (MLMI 2026) @ MICCAI

Segmentation of curvilinear anatomical structures in 3D medical images remains challenging due to complex topology, severe class imbalance, weak contrast, and large variations in structure morphology. While deep learning approaches for 3D curvilinear segmentation have been proposed, they are often tailored to specific anatomies or modalities, limiting generalization across clinical settings and leaving room for improvement. Recent generative models have shown the benefits of iterative prediction for structured segmentation tasks, yet diffusion-based methods suffer from computationally expensive sampling, hindering their use on high-resolution 3D volumes. We present 3D-CurvSegFlow, a flow matching-based model for 3D curvilinear structure segmentation. The model learns a continuous transformation from a simple source distribution to the target vascular representation, enabling progressive refinement of complex curvilinear geometries with efficient inference. We evaluate our method on Three public challenging datasets covering distinct anatomies and modalities: portal vein, cerebral vessel, and coronary arteries. Using a common architecture and training strategy across all tasks, our method outperforms general-purpose and vessel-specific approaches, with strong preservation of thin branches and vascular continuity. This work not only advances the state-of-the-art in 3D curvilinear segmentation but also opens new avenues for efficient, generalizable, and clinically applicable methods in medical image analysis.

0.23

Semi-Supervised GAN-Based Segmentation of CTPA Scans for Pulmonary Emboli Boundary Extraction

cs.CV cs.LG ·2026-11-18 ·Maria Thoma, Michalis Savelonas, Foteini Malli, Dimitris Iakovidis
published similarity 0.65
Abstract · 2026 IEEE 39th International Symposium on Computer-Based Medical Systems (CBMS), Limassol, Cyprus

Pulmonary embolism (PE) is a pulmonary vascular disease associated with high morbidity and mortality, where delayed diagnosis can be fatal. While computed tomography pulmonary angiography (CTPA) is the clinical gold standard for PE detection, interpreting these scans manually is time-consuming and error-prone. In this work, we propose a semi-supervised segmentation method, employing generative adversarial networks (GANs) to leverage large pools of unlabeled CTPA scans. The proposed method addresses more effectively the challenges of CTPA image segmentation than previous methods by employing the dice loss and the cross-entropy loss to tackle the highly imbalanced nature of CTPA scans, the feature matching loss to encourage the generator to produce anatomically plausible segmentation maps that are indistinguishable from ground truth, as well as the self-training loss to reinforce learning from highly-confident pseudo-labeled samples. The experimental evaluation on a publicly available dataset shows that the proposed method maintains accurate segmentation, despite a limited number of labeled CTPA scans. with a Dice coefficient of 88.75% and a 95th percentile Hausdorff distance (HD95) of 1.74, outperforming fully supervised baselines.

0.27

Beyond In-Distribution Metrics: A Systematic Out-of-Distribution Evaluation of Congenital Heart Disease Segmentation

cs.CV cs.AI ·2026-09-15 ·Aniketh Vijesh, Shrisharanyan Vasu, Abhijit Ramesh, Clare Pomeroy-Ward, Harikrishnan Anil Maya +3 more
published similarity 0.65
Abstract · 12 pages, 6 figures, 2 tables. Accepted at STACOM 2026, held in conjunction with MICCAI 2026

Congenital heart disease (CHD) diagnosis and surgical planning often require patient-specific 3D anatomical models, but manual segmentation is labor-intensive, particularly in complex anatomies. Although deep-learning methods can automate this process, they are typically evaluated in-distribution, despite clinically relevant shifts in scanner, protocol, institution, population, and imaging modality. We present, to our knowledge, the first systematic evaluation of out-of-distribution (OOD) generalization in CHD segmentation, using ImageCHD as a held-out target cohort. We compare representative segmentation architectures under combined CT and CMR training, CT-only training, self-supervised pretraining, and limited target-domain adaptation. In-distribution performance proves to be a poor indicator of cross-cohort robustness: nnU-Net achieves the highest validation Dice (0.77) but falls to 0.51 on ImageCHD, while SwinUNETR generalizes substantially better, reaching 0.67 Dice. MAE and JEPA pretraining provide only modest additional benefit, suggesting that architecture contributes more to robustness than the tested pretraining strategies in this setting. When limited target-domain supervision is introduced, all SwinUNETR variants exceed 0.76 Dice with only 11 labeled ImageCHD cases. These findings demonstrate that conventional in-distribution evaluation can obscure clinically important generalization failures and support explicit cross-dataset testing as a key component of CHD segmentation evaluation.

0.28