CS Paper Radar

ECG Mirage: Revealing and Mitigating the Underutilisation of ECGs in Vision-Language Models for Clinical Prediction

Jinning Liang, Mingcheng Zhu, Tingting Zhu

cs.AI ·submitted 2026-09-18 13:27 UTC · v1

Abstract

Emergency department (ED) decision-making relies on heterogeneous clinical information, including patient history, vital signs, laboratory results, and electrocardiograms (ECGs). Vision--language models (VLMs) can jointly process these modalities, but strong predictive performance does not necessarily imply meaningful use of the correct patient's ECG. We term this failure mode ECG Mirage: apparent multimodal capability without useful dependence on patient-specific ECG information. We distinguish two forms: ECG neglect, where ECGs provide little predictive benefit, and ECG confusion, where matched ECGs outperform no-image inputs but not mismatched ECGs. To evaluate these behaviours, we compare predictions obtained with matched ECGs, outcome-discordant mismatched ECGs, and no-image inputs while holding the clinical text and prediction targets fixed. Across four VLMs on MDS-ED, matched ECGs provide no consistent advantage for either ICU admission or clinical deterioration prediction. We then train four restricted visual prompts using supervised learning followed by conditional direct preference optimisation, while keeping the VLM backbone frozen. The resulting models achieve balanced accuracies of 70.6% for ICU admission and 67.5% for deterioration and increase the matched-versus-mismatched performance gap to approximately 16.5 and 5.5 percentage points, respectively. Overall, our study identifies ECG Mirage in multimodal clinical prediction and introduces visual prompt tuning as an efficient mitigation strategy.

Closest papers

Nearest neighbours by meaning across the whole index.

IECG: Weakly-Supervised Interpretability-Enhancing Foundation Model for Electrocardiogram

cs.AI ·2026-10-20 ·Huynh Dang Nguyen, Trong-Thang Pham, Anh-Thuan Do, Phuong-Nhi Dinh, Brijesh Patel +2 more
published similarity 0.66
Abstract · Accepted in WACV 2027

Electrocardiography (ECG) is a cornerstone of cardiac diagnosis, yet most machine learning methods for ECG analysis lack interpretability, limiting their clinical adoption. We present IECG, a weakly-supervised ECG foundation model that jointly performs diagnostic classification and abnormality localization without requiring manual abnormal region annotations, overcoming a long-standing bottleneck in interpretable ECG learning. Central to IECG is a weakly-supervised attention strategy that explicitly incorporates clinical priors by deriving abnormality heatmap pseudo-labels from established ECG clinical knowledge. These pseudo-labels guide the model's CLS-token attention through a KL-divergence loss, encouraging it to focus on diagnostically relevant waveform regions. During pretraining, IECG is jointly optimized with clinically guided attention supervision, ECG-text contrastive alignment, and report captioning objectives, enabling it to learn rich multimodal representations while producing localized waveform evidence. By explicitly aggregating features from clinically guided attention regions, IECG grounds its diagnostic predictions in interpretable waveform evidence without sacrificing classification performance. Extensive experiments demonstrate that IECG achieves state-of-the-art abnormal region localization while maintaining strong classification performance, advancing trustworthy foundation models for ECG analysis.

0.31

TRACE: Expert-Aligned ECG Representation Learning with Rigorous Benchmarking and Real-World Validation in Acute Cardiac Care

cs.LG cs.AI ·2026-09-28 ·Lovely Yeswanth Panchumarthi, Andrew Lu, Saurabh Kataria, Delgersuren Bold, Minxiao Wang +9 more
benchmark similarity 0.65
Abstract · 42 pages, 4 figures and 1 table

TRACE (Text-Reinforced Analysis of Cardio ECGs) is a multimodal electrocardiogram (ECG) representation model that learns clinically grounded signal embeddings for downstream cardiac classification. It is designed to address the limitations of existing CLIP-style training, which often struggles with noisy clinical text and fails to leverage the complementary strengths of unimodal (from ECG) and cross-modal (between ECG and matched cardiologist reports) learning. To bridge this gap, we propose a hybrid architecture that jointly learns unimodal and cross-modal representations via uncertainty-weighted multi-task learning while utilizing an LLM-based pipeline to extract high-fidelity findings from cardiologist reports. We evaluate TRACE across a spectrum of clinical urgency, establishing robust performance on public benchmarks for arrhythmia classification and structural abnormalities relative to existing unimodal and multimodal ECG models. To demonstrate real-world utility, we further validate the model on acute coronary occlusion (ACO), where the prevailing ST-elevation criteria miss 25-34% of true occlusions. Utilizing a large private ACO dataset with expert-annotated ground truth, TRACE significantly outperforms real-world clinical practice, yielding a 19.0% increase in sensitivity or a 62.6% reduction in false positive rates at the clinical baseline. This extensive evaluation confirms that TRACE delivers both strong performance on benchmark tasks and tangible clinical impact in the most acute, high-risk cardiac scenarios.

0.11

Rethinking Correctness for Uncertainty Estimation in Clinical Prediction with Vision-Language Models

cs.LG ·2026-09-14 ·Mingcheng Zhu, Jinning Liang, Tingting Zhu
similarity 0.65
Abstract · 5 pages, 3 figures

Vision-language models are increasingly explored for clinical prediction from electronic health records and medical images, where identifying unreliable predictions is important for safe deployment. Uncertainty estimation (UE) enables detecting such predictions, but its evaluation depends on a correctness criterion that determines whether each model output is correct. If this criterion disagrees with human judgement or distorts downstream UE performance, conclusions about model reliability can be misleading. We introduce a two-axis framework that evaluates correctness criteria by their agreement with human judgements and fidelity to human-referenced UE performance. We assess eight criteria across three clinical prediction tasks and three models using 450 predictions annotated by two reviewers. Across the audited tasks, canonical exact matching (EM) achieved the highest observed human agreement and lowest UE distortion, while the BERT-based matching (BEM) and LLM-judge also showed strong human agreement. Across four UE methods and 23,254 clinical predictions, criterion choice changed error-detection AUROC by up to 0.146 and reversed the relative ranking of UE methods. The LLM-judge also selectively accepted invalid or uncertain outputs, accepting 16 of 30 such human-identified errors. These results demonstrate that correctness assessment is an integral component of clinical UE evaluation and should be validated before UE methods are compared.

0.18

ViCoR: Estimating Visual Necessity via Counterfactual Residuals for Multimodal Medical Data Selection

cs.LG cs.AI ·2026-10-18 ·Haotian Gan, Yudong Li, Xintian Li, Jiarui Han, Weidong Tang
top venuebenchmarkcode/data similarity 0.65
Abstract · Accepted to NeurIPS2026

Medical vision-language models often exhibit hidden stratification, where high overall performance masks failures on subtle visual findings and overlapping clinical conditions. We trace this failure mode to modality imbalance during fine-tuning, where strong linguistic priors can suppress visually grounded learning and induce plausible but weakly grounded rationales. Existing data selection methods usually score image-text pairs as holistic units, without separating visually necessary samples from shortcut-solvable ones. To address this problem, we propose ViCoR, a counterfactual-residual-based data selection method for Med-VLM fine-tuning. ViCoR estimates visual necessity through image-dependent optimization residuals, calibrates the resulting scores with medical supervision compatibility, and constructs a diversity-aware fine-tuning subset. Across two backbones, four external generalization benchmarks, and one multi-disease robustness benchmark, ViCoR-selected 20\% subsets achieve the strongest average performance compared with full-data fine-tuning and selection baselines. Gradient-level analysis and perturbation audits further show that ViCoR enriches image-dependent training signals and strengthens image-level and evidence-region dependence. The code will be available at https://github.com/htgan-ai/ViCoR.

0.40

Vision-language classification of Torsades de Pointes risk from pseudo-coloured ECG images: do T-wave morphology and colour context help or hinder?

cs.CV ·2026-11-01 ·Dario Labrador Alonso, Alaa Alahmadi
similarity 0.65
Abstract

Contextual prompting is widely presented as a cheap route to better performance from vision models, but the evidence comes largely from benchmarks in which an attribute named in a prompt is also present in the image, confounding the two channels. We investigate the significance of context for a semantically complex biomedical signal classification task using a domain-adapted vision-language model, BiomedCLIP. Building on our previous work that renders electrocardiogram (ECG) signals as colour-enhanced images to flag Torsades de Pointes risk using the QT-nomogram and pseudo-colouring technique, we cross four context strategies with four T-wave morphology label sets, evaluated zero-shot and with a class-balanced linear probe, under five-fold stratified cross-validation on 9,789 images. Zero-shot inference is unusable. Once trained, the colour-enhanced images outperform the colourless ones on every metric: balanced accuracy 0.903 versus 0.825, sensitivity 0.914 versus 0.827, specificity 0.892 versus 0.823, ROC-AUC 0.955 versus 0.888 and Matthews correlation coefficient (MCC) 0.437 versus 0.311. Crossing colour-in-image with colour-in-prompt separates the confounded channels: the image contributes +0.126 MCC (+0.078 balanced accuracy, +0.067 ROC-AUC) whether or not the prompt names colour, whereas naming it changes nothing under any metric. Context is two-sided: the best prompt set gains 0.008 MCC where colour is present and 0.016 where it is not, while the worst loses 0.017 and 0.013. For a signal rendered as an image, the leverage lies in how the signal is drawn rather than in how it is described - and where the endpoint is an interval measurement, prompts naming T-wave morphology describe the wrong quantity.

0.22

ECGJev: Question-Conditioned Option Scoring for 12-Lead ECG Interpretation

cs.LG eess.SP ·2026-11-01 ·Zhuoyuan Li, Xidong Wu, Hanyu Cui, Jiansong Gao, Rongrong Fu +2 more
code/datahot direction similarity 0.65
Abstract · 5 pages (4 pages plus ethics statement and references), 2 figures. Submitted to IEEE ISBI 2027. Zhuoyuan Li and Xidong Wu contributed equall…

Clinical interpretation of a 12-lead electrocardiogram (ECG) often requires answering targeted diagnostic questions with clear, quantifiable decisions. We introduce ECGJev, the first ECG framework built on a decision decoder, a pretrained language model trained to return a probability for every listed option. A question-conditioned bridge compresses frozen waveform and spectrogram features into 16 ECG tokens, and a two-stage procedure adapts the bridge and low-rank adapters on the decoder's native option readout using diagnostic labels alone, scoring all options for a question in one forward pass. Evaluated across four ECG benchmarks, ECGJev achieves the highest macro-F1 on CPSC2018 (80.46%), PTB-XL (71.68%) and Chapman (96.86%), and generalizes competitively to the small-sample PTB cohort. Concept probes and perturbation analyses further show that its predictions rely directly on the ECG input, remain stable under option-order and wording variations, and transfer zero-shot to unseen waveform concepts. Our code is available at https://github.com/zhuoyuanli-lab/ECGJev.

0.34

Target Recoverability under Selective Labels: A Failure Mode in ECG-Based Acute Coronary Syndrome Prediction

cs.LG cs.AI ·2026-11-08 ·Alexander Schubert, Anton Håkansson, Ziad Obermeyer
top venue similarity 0.65
Abstract · Accepted to the Proceedings of Machine Learning for Health (ML4H) 2026

Medical machine learning models often learn from diagnoses observed only in patients clinicians chose to test. The biases of such selective labels are well known, but less understood is what happens when a diagnosis includes cases the model's inputs cannot detect. Because testing decisions draw on information the model does not see, these cases can appear predictable among tested patients, producing relationships that fail, and can even reverse, among the untested. We formalize this mechanism and show in simulations that it worsens as testing becomes more selective or undetectable cases more common, while models trained only on detectable cases remain informative. We empirically study electrocardiogram (ECG)-based prediction of acute coronary syndrome (ACS), which is confirmed only by invasive testing and whose subtypes differ in how clearly they appear on the ECG. Three ECG foundation models fine-tuned on revascularization, our ACS proxy, performed well among catheterized patients (AUROC 0.740-0.770) but worse than chance for 30-day major adverse cardiovascular events among untested patients (AUROC 0.381-0.478). Models trained on ECG-visible subtypes retained discrimination (AUROC 0.680-0.709), whereas a model trained on unstable angina, which can occur without ECG changes, was inversely related to risk (AUROC 0.310). Under selective labels, effective learning therefore requires prediction targets that can be inferred from the model's inputs and evaluation in the intended deployment population, which often extends beyond the patients who were tested.

0.31

PACE-ECG: Patient-Aware Contrastive ECG Pretraining; Leveraging Longitudinal Structure for Transferable Representations

cs.AI ·2026-11-09 ·Riccardo Cavarra, Lupo Lovatelli, Shaheim Ogbomo-Harmitt, Jakub Grzelak, Adelaide De Vecchi +2 more
similarity 0.65
Abstract

Foundation models for physiological signals are commonly evaluated on in-domain interpretation tasks, but their ability to transfer to longitudinal clinical prognosis across healthcare systems remains uncertain. We developed PACE-ECG, an ECG pretraining framework that combines contrastive learning across temporally neighbouring recordings from the same patient with clinically guided multitask supervision. We evaluated frozen ECG representations for post-myocardial-infarction all-cause mortality and incident heart failure prediction across MIMIC-IV and two external HEEDB cohorts, comparing the proposed representation with DeepECG-SSL, a state-of-the-art model, and two control baselines. Across both downstream tasks, the proposed model outperformed DeepECG-SSL in-domain (AUROC +1.9% for mortality and +4.0% for heart failure) and on external populations in HEEDB-Boston (+5.6% for mortality +10.3% for heart failure) and HEEDB-Atlanta (+6.9% for mortality +11.0% for heart failure) while using an encoder 41 times smaller and a fraction of the pretraining data. A matched ablation showed that patient-temporal pairing improved external mortality and heart failure AUROC over generic self-supervision by 7.0% and 9.4% on HEEDB-Boston and 8.9% and 9.3% on HEEDB-Atlanta. These findings indicate that pretraining-objective design greatly influences downstream transfer alongside model and data scale, and that in-domain performance alone is insufficient to establish the clinical generalisability of ECG foundation representations.

0.17