CS Paper Radar

Reading the Mask: Descriptor-Grounded BI-RADS Rationale Generation for Breast MRI Segmentation

Hrishikesh Singh, Aarti Sharma, Divyanshu Kumar, Millie Pant

cs.CV cs.AI ·submitted 2026-10-21 07:50 UTC · v1

Abstract

Deep learning based models accurately delineate tumours on dynamic contrast-enhanced MRI (DCE-MRI), but segmentation masks alone do not provide the rationales or reasoning used by radiologists to assess the lesion suspicion. This limits the explainability as well the clinical integration. We investigate whether clinically relevant descriptors can instead be derived from the segmentation masks and their dynamic contrast-enhancement series. We have used Breast Imaging Reporting and Data System (BI-RADS) guidelines as a structured framework. Using these guidelines we designed a BI-RADS decision tree with eight quantitative measurements using radiomics, together with the lesion size and component count, that determine the descriptors at eight nodes. Using these descriptors we generate natural-language rationale using a large language model. The model was constrained to express only the derived records from the decision tree. After applying this framework to the expert and automatic masks of each of the 1,506 cases in the MAMA-MIA dataset, we generated a reproducible corpus of 3,012 BI-RADS grounded rationales. We obtained a faithfulness score of 99.98% and a counterfactual accuracy of 99.3%, indicating that the generated rationales faithfully verbalises the measured descriptors and correctly reflects changes in the underlying record. Agreement between the descriptors from expert and automatic masks falls from 0.832 to 0.549 as mask quality falls from Good to Missed. The generated rationales are intended as draft representations for radiologist review and final sign-off rather than autonomous clinical reports.

Closest papers

Nearest neighbours by meaning across the whole index.

A Concept-informed Interpretable Artificial Intelligence Framework for Breast Ultrasound Diagnosis

cs.CV ·2026-10-12 ·Shengzhou Zhong, Ziying Wang, Liang Li, Lihua Yan, Shuochun Chen +12 more
code/data similarity 0.72
Abstract · 31 pages, 17 figures, 4 tables

Most existing artificial intelligence systems for breast ultrasound rely primarily on image-driven prediction, with limited incorporation of the knowledge from the Breast Imaging Reporting and Data System (BI-RADS) and the intermediate assessment processes underlying radiologists' workflow, thereby constraining their interpretability and alignment with clinical practice. We curated, to our knowledge, the largest multicenter collection of breast ultrasound images paired with structured BI-RADS descriptors, comprising 9,941 patients across 20 independent datasets. Across 14 testing datasets, IBUS demonstrated consistent generalization performance across different clinical centers and imaging conditions. Incorporating BI-RADS descriptors enabled fine-grained risk stratification, while cross-checking predicted BI-RADS grades against malignancy probabilities identified internally discordant predictions that may warrant additional clinical review. In a reader study involving eight radiologists, IBUS assistance increased diagnostic sensitivity among both junior and senior readers, with the potential of guiding radiologists to make targeted revision in risk stratification. Overall, IBUS exhibits its potential as a second-stage decision-support system that complements radiologists by providing additional risk estimates and interpretable evidence for diagnostically challenging cases. The demo video of IBUS platform is publicly available at https://github.com/zsz1997/IBUS.

0.30

Explainable Multimodal AI Approach to Reduce Unnecessary Breast Biopsies

cs.AI ·2026-10-26 ·Sadam Hussain, Mansoor Ali, Daly Betzabeth Avendaño-Avalos, Jose Gerardo Tamez-Peña
similarity 0.69
Abstract

Recently, there has been rise in multimodal vision-language models for computer-aided diagnosis (CAD), yet their application in breast diagnosis, particularly for predicting biopsy outcomes remains limited. This study presents a multimodal vision-language approach, incorporating Grad-CAM as an Explainable AI (XAI) technique, to predict benign tumors in patients recommended for biopsy, especially those classified as BI-RADS 4 and 5. A novel multimodal framework was devel- oped using Vision Transformer (ViT) for imaging feature extraction and DeBERTa-V3-Large for textual embeddings. The model was trained and evaluated on a comprehensive in-house dataset comprising 1,632 patients, each with four mammography views and a corresponding radiology report. The features of both modalities were extracted independently and fused by late fusion before classification. Grad-CAM was used to generate visual explanations highlighting key regions influencing model predictions. The proposed model demonstrated high sensitivity 90%, 95%, 99%, and 100% with corresponding specificities of 40%, 36%, 18%, and 11%, respectively. The best AUC achieved was 0.72 by multimodal approach on the test set, outperforming the image only (0.66) and text-only (0.68) baselines. The proposed multimodal architecture, enhanced with Grad-CAM, demonstrates a strong potential to identify benign tumors among high-risk patients, offering interpretable insights and thereby helping reduce unnecessary biopsies, patient anxiety, and healthcare costs.

0.10

Interpretable Deep Learning Models for Breast Cancer Diagnosis Using Medical Imaging

cs.CV cs.AI ·2026-10-27 ·Sami Alnawali, Muhammad Atif Saeed, Abdul Hadi, Azka Atiq, Akhtar Jamil +1 more
benchmark similarity 0.68
Abstract · 37 page, 45 Figures

Deep learning shows promise for breast cancer diagnosis, but limited interpretability hinders clinical adoption. We benchmark ten ImageNet-pretrained models (ResNet50, VGG19, InceptionV3, DenseNet121, Xception, MobileNetV2, EfficientNet, ConvNeXt, ViT, and Swin Transformer) for benign-versus-malignant classification on the BreakHis histopathology dataset using a patient-level split, and explain their predictions with Grad-CAM, LIME, Integrated Gradients, and LRP. DenseNet121 performed best, with 99.4% test accuracy and an AUC-ROC of 0.999, and significantly outperformed all other models (p < 0.001). Transformer-based models trailed the CNNs. The explanations consistently highlighted nuclei-rich and glandular regions of diagnostic relevance. Combining transfer learning with explainable AI can thus yield accurate, interpretable tools for breast cancer diagnosis.

0.19

AMPLIFAI: A Multiphase CT Dataset for Benchmarking Clinical Reasoning in LI-RADS Assessment of Liver Lesions

cs.CV cs.LG ·2026-08-14 ·Pranav Kulkarni, Nikhil Shah, Amritansh Suryavanshi, Jana G. Delfino, James Tonascia +7 more
dataset similarity 0.67
Abstract · 15 pages, 6 figures, 3 tables

Hepatocellular carcinoma (HCC) is the third leading cause of cancer-related mortality worldwide, with early detection improving survival from <20% to >70%. The standardized Liver Imaging Reporting and Data System (LI-RADS) criteria provide an imaging-based diagnostic framework to evaluate liver lesions for HCC, serving as a foundation for automating HCC detection with artificial intelligence (AI). However, the lack of large, publicly available datasets with high-quality annotations has limited the development and evaluation of AI models for automated LI-RADS assessment. We introduce AMPLIFAI dataset, the first public dataset of 590 multiphase abdominal CT studies annotated with LI-RADS categories, lesion size, and voxel-level segmentations for three major LI-RADS features: arterial phase hyperenhancement, washout, and enhancing capsule. The dataset was curated and harmonized from four public datasets and augmented with expert annotations from five board-certified radiologists and one resident. Following the Datasheets for Datasets format, this paper details the dataset's composition, curation and harmonization process, and annotation workflow to support transparent, reproducible research in medical imaging AI.

0.10

Deep Learning-Based Prediction of HER2 Status from Whole-Brain MRI in Breast Cancer Brain Metastases

cs.LG ·2026-10-23 ·Sara Early, Chris Kang, Vibujithan Vigneshwaran, Kimberly Amador, Roger Y. Tsang +1 more
similarity 0.67
Abstract

Purpose: Human epidermal growth factor receptor 2 (HER2) status is a key biomarker relevant for prognostication and treatment decision making in metastatic breast cancer. Thus, the aim of this work was to explore whether HER2 status can be predicted directly from whole-brain post-contrast T1-weighted (T1w) MRI scans in breast cancer brain metastases (BCBM) using deep learning, and whether providing explicit tumour segmentation information improves prediction. Methods: Using the publicly available BCBM-Radiogenomics dataset, we analysed 135 patients, including 70 HER2+ and 65 HER2- patients. Three approaches were implemented and evaluated: a conventional radiomics model, a simple fully convolutional network (SFCN) using T1w MRI scans alone, and an SFCN using T1w MRI scans with an expert-derived tumour mask as an additional input channel. Results: The radiomics model achieved an area under the receiver operating characteristic curve (ROC-AUC) of 0.602 (95% confidence interval [CI]: 0.528--0.677) while the SFCN achieved a ROC-AUC of 0.667 (95% CI, 0.63--0.71) using T1w MRI scans alone, which increased to 0.692 (95% CI, 0.58--0.81) when an explicit tumour mask was additionally provided. Conclusion: However, the modest performance across all approaches demonstrates the difficulty of predicting HER2 status from structural MRI scans alone. The difference between the segmentation-free and mask-guided SFCN models suggests that much of the predictive signal captured by the deep-learning model may be learned directly from whole-brain T1w MRI without requiring explicit tumour delineation. These findings support further investigation of segmentation-free deep-learning approaches as a potential non-invasive strategy to support HER2 status determination in BCBM.

0.19

Carving Radiology Reports at Their Evidential Joints: Anatomical Region Sets as Generation Units

cs.CL cs.CV ·2026-10-21 ·Jinwei Yang, Junhao Chen, Jianwei Zhao, Zhonghan Xu, Hong Cheng +1 more
similarity 0.67
Abstract · 28 pages, 4 figures

Organizing radiology reports into independently generated units offers a way to generate fine-grained clinical content and bind it to visual evidence. However, existing methods typically adopt generation-unit structures not directly specified in natural reports or their annotations, introducing additional modeling compromises. We propose RadCarve, which instead defines generation units directly from the anatomical region sets in existing statement-region correspondences, unifying unit formation with visual evidence selection. RadCarve's planner uses composition-specialized query banks to jointly predict unit existence and region composition from the image, iteratively refining both with regional evidence aggregated according to current membership predictions. The generator independently produces each unit's text conditioned on the joint visual evidence of its predicted region set. Compared with strong baselines, RadCarve improves CXRFEScore by 9.4% in unit generation given region sets and achieves 2.4 times the end-to-end binding F1. It also achieves leading report-generation results on Chest ImaGenome and PadChest-GR, using fewer than one-sixth as many parameters as large-scale baselines.

0.28

Feature Space Guidance for Breast Cancer Classification in DCE-MRI

cs.CV cs.AI ·2026-09-26 ·Benjamin Hamm, Yannick Kirchhoff, Maximilian Rokuss, Moritz Langenberg, Constantin Ulrich +4 more
top venuecode/data similarity 0.66
Abstract · 11 pages, 2 figures, 2 tables. Code: https://github.com/MIC-DKFZ/CURIAtor

Dynamic contrast enhanced breast MRI (DCE-MRI) is a powerful clinical tool for breast cancer detection, providing high resolution anatomical detail together with rich temporal contrast information. However, high dimensional 4D inputs, small lesions, and heterogeneous acquisition protocols across clinical sites hinder robust automated classification of healthy, benign, and malignant cases. To address these challenges, we propose a framework that dynamically analyzes latent representations to adapt to protocol-specific characteristics. Spatial variability is mitigated by reducing confounding background uptake and compensating for misalignment caused by deformable soft tissue. Additionally, relationships in the latent space across phases are leveraged to select the most informative temporal features, improving robustness to protocol-specific temporal variability. Finally, task specific discriminative features are promoted through large scale supervised lesion segmentation pretraining, which substantially enhances downstream finetuning. Evaluated under leave-one-center-out validation on the ODELIA dataset and the held-out AMBL cohort, the proposed framework substantially outperforms finetuned radiology foundation models and prior methods, improving mean AUROC by nearly 8 points and balanced accuracy by 4 points over the strongest baseline. Additionally, our method achieved first place in the MICCAI ODELIA Breast MRI Challenge 2025, further demonstrating its effectiveness for robust breast cancer classification. We publicly release our codebase under https://github.com/MIC-DKFZ/CURIAtor.

0.70

Boot-and-Feedback Framework for Generalist-Expert Model Collaboration in Breast Ultrasound Diagnosis

cs.CV cs.MM ·2026-08-25 ·Ming Cheng, Hongyu Sun, Zhaolin Chen, Jun Liu, Hossein Rahmani +1 more
published similarity 0.66
Abstract · 5 pages, 2 figures. Published in ICASSP 2026

Breast ultrasound (BUS) is widely used for breast cancer diagnosis yet remains operator-dependent. While deep learning shows promise, ensuring diagnostic reliability and interpretability is challenging. Recent Multimodal Large Language Models (MLLMs) often generate spurious descriptions due to limited domain knowledge, which mislead downstream expert models and compromise clinical validity. To address these challenges, we propose the Boot-and-Feedback (BooF) model collaboration framework for synergistic MLLM-expert interaction. Specifically, in the Boot Stage, the MLLM is guided by the BI-RADS lexicon and preliminary benign-malignant vision-expert predictions, enabling it to transfer general reasoning to BUS analysis while avoiding hallucinations. Subsequently, the Feedback Stage integrates these descriptions with visual features via a lightweight Attention-Gated Cross-Modality Fusion Module. This allows the expert to leverage textual feedback while adaptively filtering noise. Extensive experiments on multiple BUS datasets demonstrate that BooF substantially outperforms state-of-the-art methods in terms of diagnostic accuracy and interpretability.

0.52