CS Paper Radar

Vision-language classification of Torsades de Pointes risk from pseudo-coloured ECG images: do T-wave morphology and colour context help or hinder?

Dario Labrador Alonso, Alaa Alahmadi

cs.CV ·submitted 2026-11-01 14:02 UTC · v1

Abstract

Contextual prompting is widely presented as a cheap route to better performance from vision models, but the evidence comes largely from benchmarks in which an attribute named in a prompt is also present in the image, confounding the two channels. We investigate the significance of context for a semantically complex biomedical signal classification task using a domain-adapted vision-language model, BiomedCLIP. Building on our previous work that renders electrocardiogram (ECG) signals as colour-enhanced images to flag Torsades de Pointes risk using the QT-nomogram and pseudo-colouring technique, we cross four context strategies with four T-wave morphology label sets, evaluated zero-shot and with a class-balanced linear probe, under five-fold stratified cross-validation on 9,789 images. Zero-shot inference is unusable. Once trained, the colour-enhanced images outperform the colourless ones on every metric: balanced accuracy 0.903 versus 0.825, sensitivity 0.914 versus 0.827, specificity 0.892 versus 0.823, ROC-AUC 0.955 versus 0.888 and Matthews correlation coefficient (MCC) 0.437 versus 0.311. Crossing colour-in-image with colour-in-prompt separates the confounded channels: the image contributes +0.126 MCC (+0.078 balanced accuracy, +0.067 ROC-AUC) whether or not the prompt names colour, whereas naming it changes nothing under any metric. Context is two-sided: the best prompt set gains 0.008 MCC where colour is present and 0.016 where it is not, while the worst loses 0.017 and 0.013. For a signal rendered as an image, the leverage lies in how the signal is drawn rather than in how it is described - and where the endpoint is an interval measurement, prompts naming T-wave morphology describe the wrong quantity.

Closest papers

Nearest neighbours by meaning across the whole index.

Clinically Informed Colour Encoding Improves the Localisation and Stability of Explanations in Deep Learning ECG Models: External Validation in a UK Biobank Epilepsy Cohort

cs.CV ·2026-10-30 ·Lauren Hutchinson, Moses Nkgari, Alaa Alahmadi
hot direction similarity 0.71
Abstract

Objective: It is not known whether the input representation governs the quality of explanations of deep learning models for electrocardiogram analysis. Risk of dangerous ventricular repolarisation is judged from the QT-interval and the shape of the T-wave, features that are hard to assess visually and harder still to verify in a model's reasoning. We evaluate a pseudo-colour encoding filling the area beneath the QT interval, its colour sequence designed from clinical electrophysiology knowledge and mapped to a validated risk prediction model, the QT-nomogram. Methods: Two identically configured residual networks were trained on single-heartbeat images from a clinical trial, differing only in whether pseudo-colour encoding was applied, so that any difference is attributable to the chromatic channel alone. To externally validate the effect, both models were applied without fine-tuning or threshold tuning to a previously unseen epilepsy cohort from the UK-Biobank at two visits, and eight explanation methods compared on faithfulness, localisation and stability. Results: The pseudo-colour model achieved areas under the receiver operating characteristic curve of 0.903 and 0.914 at the first and follow-up visits, against 0.719 and 0.720 for the monochrome model, recovering three to five times as many at-risk heartbeats at comparable specificity. Pseudo-colour encoding improved explanation localisation and stability for almost every method, and the improvement reproduced across both visits. Conclusion: The input representation is a controllable and previously overlooked determinant of both predictive performance and explanation quality. Significance: Treating interpretability as a representation-design problem offers a route towards explanations clinicians can check against known electrophysiology.

0.31

Diagnosing as Cardiologists Do: ECG Agents with Doctor-Grounded Priors for Clinical Reasoning Across Diseases and Populations

eess.SP cs.CVcs.LG ·2026-08-10 ·Hongxiang Gao, He-yang Xu, Yuwen Li, Minghui Zhao, Zhipeng Cai +4 more
benchmark similarity 0.67
Abstract

Cardiologists interpret electrocardiograms by localizing waveform components, measuring rhythm and interval patterns, and translating these structured observations into diagnostic evidence. Whether this expert reading process can serve as an effective prior for ECG agents remains unclear. To address this question, we introduce LuminaECG, a clinically structured ECG reasoning framework that reformulates ECG interpretation as measurement-grounded visual reading. ECG signals are rendered on standard electrocardiographic grid paper to preserve the spatial and scale cues used in clinical reading. P-wave, QRS-complex, and T-wave boundaries are explicitly delineated, and color-coded segmentation decomposes the waveform into discrete visual measurement primitives. A general 2B vision-language backbone is then trained with low-rank supervised fine-tuning to associate these primitives with diagnostic reasoning, without architectural modification. Across open, proprietary, and ECG-specialist zero-shot baselines, LuminaECG improves both waveform measurement and diagnostic recovery. It reaches a clinically meaningful reader tier on the CODE-test benchmark, transfers across geographically diverse ECG datasets without retraining, and generates reports whose structure contains an emergent prognostic signal. These findings suggest that effective ECG agents require not only larger models, but supervision that preserves the alignment between measurable waveform evidence and clinical knowledge.

0.04

ECGQuest: Benchmarking and Fine-Tuning Language Models for Electrocardiography

cs.CL cs.IR ·2026-08-31 ·Mohammadsina Hassannia, Matthew A. Reyna, Reza Sameni
benchmarkcode/data similarity 0.67
Abstract

Electrocardiogram (ECG) interpretation requires knowledge of cardiology, electrophysiology, clinical diagnosis, ECG waveforms, signal acquisition, and instrumentation. Existing language-model benchmarks, however, primarily assess broad medical knowledge or interpretation of individual ECG signals and images rather than the broader contextual knowledge required for ECG interpretation. We developed ECGQuest, a literature-grounded resource for evaluating and fine-tuning ECG-specific language models. A GPT-4o-based pipeline generated questions from 23 ECG references and Computing in Cardiology proceedings from 2003-2025. The final dataset contains 10,904 unique True/False questions paired with their negated forms (21,808 Q&A pairs). We evaluated three commercial and 20 open-source language models on a held-out test set in a zero-shot setting. Five open-source models with 7-14B parameters were fine-tuned using Low-Rank Adaptation, with BERT and BiomedBERT included as supervised encoder baselines. Generalization was assessed on ECG-related subsets of MedMCQA and MedQA converted to binary True/False questions using official answer keys. Zero-shot accuracy on ECGQuest ranged from 49.5% to 74.4%, with GPT-5 performing best. General-purpose models outperformed medically specialized models, several models showed strong True/False bias, and encoder baselines performed near chance. Fine-tuning improved all open-source models by 6.5-14.1%. Fine-tuned DeepSeek-R1-Distill-Qwen-14B reached 76.3% accuracy, while a five-model voting ensemble reached 78.5%. On MedMCQA and MedQA, fine-tuning mainly benefited weaker or class-biased models and did not consistently improve strong base models. ECGQuest provides a reproducible benchmark for contextual ECG knowledge and shows that parameter-efficient fine-tuning can make smaller language models competitive with substantially larger commercial models.

0.35

From ECG Signals to Representative-Morphology Heatmaps for Biometric Recognition

cs.CV eess.SP ·2026-09-23 ·Athanasios Angelakis, Marta Gomez-Barrero
similarity 0.66
Abstract · 31 pages, 7 figures, 14 tables

Electrocardiography (ECG) contains subject-specific morphology that supports biometric recognition, yet image-based performance depends on how the waveform is rendered. We introduce representative-morphology heatmaps, a deterministic ECG-to-image representation adapted from ECGXtractor. Within each block of ten aligned beats, the five beats closest to the block mean are averaged into a 400 by L matrix and rendered either as a conventional trace or as a dense cardiac-time-by-lead heatmap. Since both representations contain identical physiological samples, their comparison isolates the effect of rendering. We evaluate verification and closed-set identification on PTB, ECG-ID, and MIMIC-IV-ECG-DEMO. Five compact models, including ZACH-ViT, are trained from scratch, while six ImageNet-pretrained CNN and transformer backbones assess model scale and visual transfer. Heatmaps improve both FNMR operating points and both identification ranks in all 15 compact model-dataset comparisons, while EER improves in 14. Across the matched experiments, EER decreases by 9.59 percentage points and Rank-1 increases by 24.69 points on average. ConvNeXt-Tiny reaches 2.43% EER on PTB and 5.79% on ECG-ID, whereas DeiT-Base reaches 14.92% on MIMIC-DEMO. ImageNet initialization clearly benefits the two multilead datasets but has a mixed effect on ECG-ID, and performance does not increase monotonically with model size. The best heatmap systems approach the strongest signal-domain EER on PTB and ECG-ID, while DeiT-Base provides the strongest evaluated performance on MIMIC-DEMO. Lead-channel ablation further shows that useful channel combinations depend on the cohort and biometric task. Overall, representative-morphology heatmaps provide an effective image representation for ECG verification and identification.

0.42

ECG Mirage: Revealing and Mitigating the Underutilisation of ECGs in Vision-Language Models for Clinical Prediction

cs.AI ·2026-09-18 ·Jinning Liang, Mingcheng Zhu, Tingting Zhu
similarity 0.65
Abstract

Emergency department (ED) decision-making relies on heterogeneous clinical information, including patient history, vital signs, laboratory results, and electrocardiograms (ECGs). Vision--language models (VLMs) can jointly process these modalities, but strong predictive performance does not necessarily imply meaningful use of the correct patient's ECG. We term this failure mode ECG Mirage: apparent multimodal capability without useful dependence on patient-specific ECG information. We distinguish two forms: ECG neglect, where ECGs provide little predictive benefit, and ECG confusion, where matched ECGs outperform no-image inputs but not mismatched ECGs. To evaluate these behaviours, we compare predictions obtained with matched ECGs, outcome-discordant mismatched ECGs, and no-image inputs while holding the clinical text and prediction targets fixed. Across four VLMs on MDS-ED, matched ECGs provide no consistent advantage for either ICU admission or clinical deterioration prediction. We then train four restricted visual prompts using supervised learning followed by conditional direct preference optimisation, while keeping the VLM backbone frozen. The resulting models achieve balanced accuracies of 70.6% for ICU admission and 67.5% for deterioration and increase the matched-versus-mismatched performance gap to approximately 16.5 and 5.5 percentage points, respectively. Overall, our study identifies ECG Mirage in multimodal clinical prediction and introduces visual prompt tuning as an efficient mitigation strategy.

0.38

IECG: Weakly-Supervised Interpretability-Enhancing Foundation Model for Electrocardiogram

cs.AI ·2026-10-20 ·Huynh Dang Nguyen, Trong-Thang Pham, Anh-Thuan Do, Phuong-Nhi Dinh, Brijesh Patel +2 more
published similarity 0.62
Abstract · Accepted in WACV 2027

Electrocardiography (ECG) is a cornerstone of cardiac diagnosis, yet most machine learning methods for ECG analysis lack interpretability, limiting their clinical adoption. We present IECG, a weakly-supervised ECG foundation model that jointly performs diagnostic classification and abnormality localization without requiring manual abnormal region annotations, overcoming a long-standing bottleneck in interpretable ECG learning. Central to IECG is a weakly-supervised attention strategy that explicitly incorporates clinical priors by deriving abnormality heatmap pseudo-labels from established ECG clinical knowledge. These pseudo-labels guide the model's CLS-token attention through a KL-divergence loss, encouraging it to focus on diagnostically relevant waveform regions. During pretraining, IECG is jointly optimized with clinically guided attention supervision, ECG-text contrastive alignment, and report captioning objectives, enabling it to learn rich multimodal representations while producing localized waveform evidence. By explicitly aggregating features from clinically guided attention regions, IECG grounds its diagnostic predictions in interpretable waveform evidence without sacrificing classification performance. Extensive experiments demonstrate that IECG achieves state-of-the-art abnormal region localization while maintaining strong classification performance, advancing trustworthy foundation models for ECG analysis.

0.31

Holtercare-Bench: A Multimodal Benchmark for Evaluating Long-Term Dynamic ECG Analysis

cs.LG ·2026-08-19 ·Yihan Xie, Hanwen Cui, Runze Ye, Juekai Lin, Haoyang Wang +6 more
top venuebenchmarkcode/data similarity 0.60
Abstract

While multimodal large language models (MLLMs) excel in medical applications, most of them favor static images or short-term signals. In the critical field of dynamic electrocardiograms (ECG), models struggle with complex temporal reasoning and diagnostic report generation due to a lack of high-quality datasets and benchmarks. To address this, we introduce (i) Holtercare-23K, a large-scale multimodal dynamic ECG dataset comprising 22,980 QA pairs derived from 788 clinical Holter records and featuring a novel signal-video-text tri-modal alignment. Based on this dataset, we present (ii) Holtercare-Bench, a multimodal benchmark that evaluates models on temporal localization, clinical diagnosis, and global summarization. Zero-shot evaluations of leading MLLMs reveal a significant performance gap in processing ultra-long pathological sequences. However, fine-tuning representative models yields substantial improvements. This work illuminates the limitations of current MLLMs in electrophysiology and provides a foundational benchmark for long-term medical MLLMs. Our project is available at https://github.com/ZJU4HealthCare/Holtercare-Bench.

0.47

MS-ECG-FM: Towards a More Universal Electrocardiogram Foundation Model for Health Monitoring using Multi-source Contrastive Learning

cs.LG ·2026-10-06 ·Robert A. Lewis, I-Min Chiu, Kyle Verrier, Karthik Jayaraman Raghuram, Francoise Marvel +5 more
similarity 0.59
Abstract · Andrew C. Miller, Joseph Futoma: equal contribution. 52 pages, 4 figures, 20 tables, including supplementary information

Electrocardiography (ECG) records the electrical activity of the heart, aiding diagnosis by detecting abnormalities in cardiac function. ECG foundation models have demonstrated promising results, but are limited by a reliance on ECG interpretation reports as their sole supervision. Because interpretation reports only capture the subset of waveform information routinely recognized by clinicians, this constrains representation learning to overlook the broader diagnostic signals present in ECG. We introduce a new ECG foundation model --- MS-ECG-FM --- that is trained through contrastive alignment to multiple distinct clinical note types, including ECG, echocardiography, radiology, and discharge reports. We evaluate MS-ECG-FM on an extended set of ECG detection benchmarks, showing that it comprehensively outperforms existing methods on the full span of conditions that ECG can detect, including in reduced-lead configurations. Different reports improve representations for different diagnostic domains, while multi-source alignment captures their complementary information and produces consistently strong representations across clinically diverse tasks.

0.32