CS Paper Radar

Foundation model embeddings capture pre-diagnostic changes on screening mammograms

Kalina P. Slavkova, Eric Brattain, Aditya Gowd, Akash Pattnaik, Jean-Benoit Delbrouck, Matthew Morgan, Julie Bauml, Javid Abderezaei, Khan Siddiqui

cs.CV cs.LG ·submitted 2026-09-22 15:42 UTC · v1

Abstract

Foundation model embeddings of screening mammograms may encode pre-diagnostic tissue change without task-specific adaptation. We tested whether embeddings move faster along a data-derived "cancer direction" in women later biopsied for cancer than in matched screen-negative controls, and whether this depends on pretraining domain. We studied 1,773 biopsied women (785 malignant, 988 biopsy-negative) and 1,773 matched controls, each with at least two annual screening exams before their index exam. An identical pipeline was applied to four 2D models: Mammo-CLIP (MC, out-of-distribution mammography), HOPPR (in-distribution mammography), MedImageInsight (MII, general medical imaging), and BiomedCLIP (biomedical vision-language pretraining on literature figures). Breast-level embeddings quantified longitudinal movement along the cancer direction. We compared cases and controls using a between-patient design with complementary mixed-effects analysis, and biopsied versus healthy contralateral breasts within patients. Under matched modality in MII embedding space, malignant cases drifted significantly faster than controls in the first two screening intervals preceding the index exam; biopsy-negative cases showed significance only in the first. MC differences were significant in the first interval for both biopsy groups. Within-patient comparisons showed a broadly similar pattern, with MC significance extending to the second interval in both groups and HOPPR showing significance at interval 1. BiomedCLIP showed no significant differences in either design or biopsy group. Overall, directional embedding velocity emerges as a property of clinically grounded rather than general biomedical pretraining, showing that foundation model embeddings can encode pre-diagnostic mammographic change without task-specific adaptation.

Comment
13 pages, 5 figures, supplementary info attached

Closest papers

Nearest neighbours by meaning across the whole index.

Explicitly Aligned Multi-View Breast Cancer Risk Prediction from Variable-Length Screening Histories

cs.CV cs.AI ·2026-10-12 ·Solveig Thrun, Zijun Sun, Suaiba A. Salahuddin, Kristoffer Wickstrøm, Elisabeth Wetzer +3 more
benchmarkcode/data similarity 0.67
Abstract

Accurate breast cancer risk prediction from screening mammography is essential for improved risk stratification and personalized screening. It can benefit from both temporal breast tissue changes across longitudinal examinations and complementary anatomical information from craniocaudal (CC) and mediolateral oblique (MLO) views. Leveraging longitudinal information requires identifying corresponding tissue regions across examinations, yet most approaches rely on implicit feature alignment without explicitly enforcing spatial consistency. Recent work has demonstrated the benefits of explicit temporal alignment; however, existing methods model each view independently, preventing them from exploiting complementary CC and MLO information. Furthermore, explicit approaches have been limited to a single prior examination. To fill these gaps, we propose (github.com/sot176/FlexLMV-Net), a breast-wise longitudinal multi-view framework that jointly models CC and MLO views across screening examinations while accommodating a variable number of prior examinations. FlexLMV-Net explicitly aligns longitudinal feature maps and integrates complementary views through cross-view temporal fusion and a dual-stream attention block. Evaluated on EMBED and CSAW-CC, FlexLMV-Net consistently outperforms existing approaches across breast density, cancer subtypes, and demographic subgroups, highlighting the benefits of explicit longitudinal alignment, multi-view modeling, and flexible use of available screening history. We further introduce (github.com/sot176/BCRBench), an open-source benchmark that unifies state-of-the-art single- and multi-timepoint breast cancer risk prediction models under standardized preprocessing, training, inference, and evaluation protocols, enabling fair and reproducible comparison of existing methods.

0.25

SideAge-RiskNet: Predicted Mammographic-Age Features for Future Breast Cancer Risk and Laterality Prediction

eess.IV ·2026-11-04 ·Haoyan Ding, Encheng Su, Chaowei Wu, Lixia Wang, Qian Wang +4 more
similarity 0.65
Abstract · 5 pages, 2 figures

Whether breast-specific mammographic aging provides information beyond chronological age for future cancer prediction remains unclear. We propose SideAge-RiskNet, a framework integrating predicted mammographic-age features with breast-specific representations for multi-year breast cancer risk and conditional laterality prediction. A side-aware DenseNet121 trained on never-cancer reference examinations estimates left and right breast ages. Age-adjusted residuals and bilateral age descriptors are subsequently fused with corresponding image representations, while clinical variables contribute through an image--clinical risk branch. The framework jointly predicts cumulative cancer risk over 1--5 years and the subsequently affected breast among unilateral cancer cases. Across 4,480 EMBED and 248 external examinations, pre-cancer scans demonstrated pronounced bilateral age asymmetry. SideAge-RiskNet established state-of-the-art performance in 1--5-year risk assessment over Mirai and AsymMirai with mean risk AUROCs of 81.73% on EMBED and 72.83% on CSMC. Furthermore, it enabled conditional cancer laterality prediction, achieving AUROCs of 73.26% and 82.53%, respectively. This demonstrates the critical utility of bilateral aging phenotypes in localized breast cancer risk surveillance.

0.20

Mammo-LIFE: Longitudinal Mammographic Imaging and Clinical Feature Enrichment for Post-Radiotherapy Outcome Prediction

cs.CV eess.IV ·2026-09-22 ·Farnoush Bayatmakou, Maryam Hosseini, Reza Taleei, Arash Mohammadi
similarity 0.62
Abstract

Recent advances in Artificial Intelligence (AI)-powered Computer-Aided Diagnosis (CAD) systems have substantially improved breast cancer screening, diagnosis, and prognosis. Comparatively, postradiotherapy outcome prediction using paired longitudinal mammograms has received considerably less attention. This is largely due to the limited availability of well-annotated longitudinal datasets. Longitudinal mammograms, coupled with paired pre- and post-treatment information, provide a unique opportunity to characterize treatment-induced breast tissue changes following radiotherapy. The resulting learned representations can serve as a valuable asset for advancing personalized radiotherapy planning and post-treatment management. In this context, we propose Mammo-LIFE, a patient-level multimodal framework for post-radiotherapy outcome prediction that combines longitudinal mammographic features with patient-level clinical variables. The imaging branch processes paired pre- and post-treatment mammograms acquired from the four standard views using a mammography-specific encoder adapted via Low-Rank Adaptation (LoRA). Within each view, preand post-treatment representations are explicitly compared through a longitudinal comparison module to capture treatment-related changes. The resulting view-level embeddings are then aggregated using learned view-attention pooling to form a unified patient-level mammographic representation. Selected clinical variables are subsequently combined with the image-derived prediction probability through a late-fusion strategy. To evaluate the effectiveness of combining paired longitudinal mammograms with clinical information, experiments were conducted on an in-house clinical cohort using patient-level stratified five-fold cross-validation.

0.26

Mammography Foundation Models for Opportunistic Prediction of Major Adverse Cardiovascular Events

eess.IV cs.CVcs.LG ·2026-09-16 ·Paula Feldman, Nusrat Binta Nizam, Sunwoo Kwak, Batuhan Karaman, Katerina Dodelzon +1 more
code/data similarity 0.60
Abstract

Cardiovascular disease (CVD) remains the leading cause of death among women, yet cardiovascular risk assessment often relies on clinical variables that may be missing, outdated, or unavailable in routine care. Screening mammography offers an opportunity for opportunistic cardiovascular risk stratification because it is routinely acquired and contains vascular features, including breast arterial calcifications (BAC), that are associated with cardiovascular risk and events. We evaluate whether mammography specific foundation models, originally pretrained for breast cancer-related tasks, can transfer to cardiovascular risk prediction without cardiovascular specific supervision or explicit BAC annotation. We constructed a 5-year major adverse cardiovascular event (MACE) cohort of 22,497 women linked to electronic health record outcomes, including 500 events (2.22% prevalence). The foundation models achieved AUROCs of 0.823 and 0.822 substantially exceeding an age-only model (AUROC 0.765), despite using only the screening mammogram as input, with no clinical variables. Both foundation models evaluated assigned substantially higher predicted risk to patients with radiologist-documented BAC, despite BAC never being used as a training label, and showed activation patterns consistent with vascular findings. Together, these findings suggest that mammography foundation models can recover clinically relevant cardiovascular risk information directly from mammographic pixels and suggest that screening mammography may provide an opportunistic source of cardiovascular risk information to complement conventional clinical assessment without additional imaging. Code is available in https://github.com/PauFeld/MammoCVD

0.36

Bilateral Evidence Selection and Reasoning for Mammography Vision--Language Models

cs.CV cs.AI ·2026-11-19 ·Zhengbo Zhou, Dooman Arefan, Shandong Wu
similarity 0.59
Abstract

Medical vision--language models (VLMs) often need to identify small, subtle lesions within high-resolution images containing extensive normal anatomy and redundant visual information. Visual-token compression improves efficiency, but preserving salient regions alone can discard the context needed for diagnostic interpretation. This context can extend across images: in bilateral mammography, the opposite breast provides a patient-specific anatomical reference. Yet retaining such evidence does not ensure that the model uses it effectively. We propose Bilateral Evidence Selection and Reasoning (BESR), which connects evidence preservation with comparative assessment through a bilateral-guided token selector and a bilateral reward. Under a shared token budget, the selector jointly prioritizes lesion appearance and its local and bilateral context, while the reward encourages assessments that benefit from the preserved comparison. Experiments on a bilateral dataset constructed from CBIS-DDSM and on VinDr-Mammo demonstrate improved evidence retention and diagnostic performance under constrained visual-token budgets. Inference benchmarking further shows reduced prefill latency.

0.30

A Multimodal Foundation Model for Longitudinal Patient Representation and Scalable Insight Generation in Oncology

cs.LG ·2026-08-25 ·Eugene Vorontsov, Yi Kan Wang, Alican Bozkurt, Adam Casson, Ludmila Tydlitatova +11 more
similarity 0.59
Abstract

Precision oncology necessitates a longitudinal model of patient state that captures cancer evolution and treatment over time, integrating multimodal observations. We introduce the oFM, a foundation model developed on a real-world oncology cohort of 1.67 million cancer patients that integrates clinical trajectories with DNA, RNA, and H&E pathology. Patient-level partitions were reserved for training, validation, and testing, with over one million patients used for training. The oFM encodes daily clinical and molecular episodes and, along with pathology images, integrates them over time to produce a patient state embedding. We evaluate frozen oFM embeddings against expert-curated clinical and molecular baseline features. In prognostic benchmarks, the oFM improved AUC for treatment response, progression-free survival, and overall survival (0.774 vs. 0.563 for overall survival). Across 11 comparative-treatment cohorts, the oFM embeddings achieved a three-fold higher pooled and scale-normalized treatment-benefit AUTOC than baseline features with improved benefit ranking in 9 of 11 cohorts, and provided stronger prognostic discrimination within both treatment arms. We also evaluated a mechanism discovery framework that interprets downstream models built on oFM embeddings by linking their predicted outcomes to clinically and biologically grounded mechanisms through an evidence-grounded temporal graph, enabling evaluation in clinical and drug-development applications.

0.09

Solving the Needle-in-a-Haystack Problem in Mammography Vision-Language Model with Differentiable Subset Sampling

cs.CV ·2026-09-02 ·Young Seok Jeon, Beatrice Brown-Mulry, Rohan Satya Isaac, Anjana Dissanayaka, Theo Dapamede +3 more
positioncode/data similarity 0.58
Abstract

There is growing interest in adopting CLIP-style vision--language model (VLM) pretraining for mammography. However, models that directly employ the standard CLIP architecture and training objective exhibit limited zero-shot performance in clinically important tasks such as cancer, finding-type, and BI-RADS predictions. We argue that this underwhelming performance is due to neglecting two characteristics of mammography data: (1) its high-res nature, and (2) homogeneity of radiology reports, largely driven by a predominance of negative/benign findings on examinations. We propose TopKSigLIP, a VLM designed to address these two limitations through a novel architecture and learning objectives. Instead of downscaling high-res mammography images to satisfy GPU memory constraints, TopKSigLIP introduces TopK-Patch module that learns to sample a sparse set of high-res patches likely to contain lesions, sidestepping the resolution--batch size tradeoff of VLM training. The sampled patch locations additionally serve as a built-in localization tool. To address report homogeneity, we replace the contrastive loss, which falsely repels semantically similar pairs, with a Sup-sigmoid loss. Sup-sigmoid loss extends the sigmoid loss from SigLIP with soft labels derived from structured data. TopKSigLIP outperforms existing open-source mammography and general medical VLMs on both internal and external benchmarks on density assessment, BI-RADS classification, finding subtyping, and cancer prediction under zero-shot evaluation. TopKSigLIP remains competitive under linear probing despite using a significantly smaller vision encoder and smaller training batches than baselines. The TopK-Patch module additionally achieves superior lesion localization over post-hoc Grad-CAM. Code and weights are made public:https://github.com/Youngseok0001/TopKSigLIP.

0.30

Feature Space Guidance for Breast Cancer Classification in DCE-MRI

cs.CV cs.AI ·2026-09-26 ·Benjamin Hamm, Yannick Kirchhoff, Maximilian Rokuss, Moritz Langenberg, Constantin Ulrich +4 more
top venuecode/data similarity 0.57
Abstract · 11 pages, 2 figures, 2 tables. Code: https://github.com/MIC-DKFZ/CURIAtor

Dynamic contrast enhanced breast MRI (DCE-MRI) is a powerful clinical tool for breast cancer detection, providing high resolution anatomical detail together with rich temporal contrast information. However, high dimensional 4D inputs, small lesions, and heterogeneous acquisition protocols across clinical sites hinder robust automated classification of healthy, benign, and malignant cases. To address these challenges, we propose a framework that dynamically analyzes latent representations to adapt to protocol-specific characteristics. Spatial variability is mitigated by reducing confounding background uptake and compensating for misalignment caused by deformable soft tissue. Additionally, relationships in the latent space across phases are leveraged to select the most informative temporal features, improving robustness to protocol-specific temporal variability. Finally, task specific discriminative features are promoted through large scale supervised lesion segmentation pretraining, which substantially enhances downstream finetuning. Evaluated under leave-one-center-out validation on the ODELIA dataset and the held-out AMBL cohort, the proposed framework substantially outperforms finetuned radiology foundation models and prior methods, improving mean AUROC by nearly 8 points and balanced accuracy by 4 points over the strongest baseline. Additionally, our method achieved first place in the MICCAI ODELIA Breast MRI Challenge 2025, further demonstrating its effectiveness for robust breast cancer classification. We publicly release our codebase under https://github.com/MIC-DKFZ/CURIAtor.

0.70