CS Paper Radar

Compact and Decodable Electrocardiogram Text for Large Language Models

Elif Dorukbaşı, İlker Türker

cs.CL ·submitted 2026-11-17 14:13 UTC · v1
code/data

Abstract

When a twelve-lead electrocardiogram is written as text for a large language model, the number of tokens has to be balanced against the information lost through compression. This paper presents a deterministic encoding that writes eight independent leads as amplitude letters and exact time gaps and recovers the remaining four limb leads algebraically. The key points are selected with a cardiac-cycle-dependent tolerance, the amplitudes are parameterized by a 26-letter logarithmic codebook learned from a training set, and a 10-step tolerance schedule keeps all records within a fixed token budget. The encoding was applied to 21,388 records from the public PTB-XL database and evaluated using an exact decode-and-re-encode round trip, waveform reconstruction error, and a paired comparison of convolutional classifiers trained with 5 different seeds on signals from the original and decoded databases. All records were contained in 4,080 tokens (average: 2,866 tokens, 64.3% fewer than a per-sample transcription), and all stored lead strings were exactly reproduced. The average correlation between the decoded and reference signals was 0.982. The macro-averaged area under the receiver operating characteristic curve was 0.9167 for the reference signals and 0.9157 for the decoded signals (paired difference -0.0010; 95% patient-level bootstrap interval, -0.0028 to 0.0007), within the pre-specified non-inferiority margin of -0.010; this is not a confirmatory result, because the test fold had been used during encoder development. The diagnostic content of a compact text can therefore be preserved, while the language model's ability to read the text is yet another matter.

Comment
9 pages, 5 figures, 5 tables. Preprint of a chapter submitted to "Advances in Artificial Intelligence and Data Science: Methods, Systems, and Applications" (Karabük University Press). Code: https://github.com/ekabullar/-ecg26c-llm

Closest papers

Nearest neighbours by meaning across the whole index.

Beat-Synchronous Tokenization for ECG Transformers

cs.LG eess.SP ·2026-08-31 ·Ahmed Sameh, Nolan Wilson, Max Enderlein, Yogatheesan Varatharajah
published similarity 0.66
Abstract · 6 pages, 1 figure, 3 tables. Accepted at the 2026 IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2026)

Transformer-based electrocardiogram (ECG) models commonly tokenize waveforms into fixed temporal patches. Though convenient, fixed patching can split heartbeat structures across token boundaries. We study beat-synchronous tokenization as a physiologically grounded alternative, comparing fixed patches with three beat-aligned strategies: resampled beats, adaptive pooled beats, and resampled beats augmented with R--R interval information. Experiments span two settings: 10-second 12-lead diagnostic classification on PTB-XL after MIMIC-IV-ECG masked pretraining, and 60-second single-lead rhythm classification on Icentia11k after patient-level contrastive pretraining. On PTB-XL, resampled beat tokens achieve the highest mean macro Area Under the ROC Curve (AUROC; 0.8945) and nearly match the best fixed-patch macro Area Under the Precision-Recall Curve (AUPRC; 0.7414), reducing average sequence length from 100 to 11.2 tokens. On Icentia11k, beat-synchronous tokenizers obtain comparable AUPRC to fixed patching with better stability across runs. These results suggest morphology-preserving beat tokenization is a compact, competitive alternative to fixed temporal patching.

0.20

On the role of the tokenizer in ECG transformer models

cs.LG cs.AI ·2026-09-14 ·Jiawei Li, Fabio Bonassi, Johan Sundström, Thomas B. Schön, Antônio H. Ribeiro
code/datahot direction similarity 0.65
Abstract

Tokenization determines both the physiological content presented to an ECG Transformer and the sequence over which attention operates. We compare eight tokenization strategies across Transformer, Informer, Reformer, and FEDformer on the nine-label CPSC2018 classification task. The input projection and principal backbone capacity are controlled to isolate the effect of token construction. Median-beat and HeartLang tokenization achieve mean macro-AUCs of 0.893 and 0.889 across the four backbones, compared with 0.822 and 0.824 for point-wise and patch-wise tokenization. Pooling the two physiology-aware representations yields an 8.2% relative improvement in macro-AUC. They also reduce mean sequence length from 1,250 to 158 tokens and mean peak training memory from 5.21 to 0.27 GB. The results show that aligning tokens with ECG morphology can improve both predictive performance and memory efficiency without increasing backbone capacity. The source code is available on https://github.com/LeeJarvis996/ecg_tokenizer.

0.35

ECGWmark: Deep Watermarking of Electrocardiogram Signals With Error-Correcting Codes

eess.SP cs.CRcs.CV ·2026-10-09 ·Kuan-Yu Chen, Kuan-Chen Wang, Yu Tsao, Jian-Jiun Ding
similarity 0.62
Abstract · 10 pages, 3 figures, 4 tables

Wearable electrocardiogram (ECG) recordings pass through multiple networks and systems before automated interpretation, making patient metadata vulnerable to loss. Digital watermarking embeds this metadata directly into the signal. However, conventional ECG watermarking methods fail under temporal desynchronization, and waveform similarity does not guarantee downstream diagnostic preservation. This article presents ECGWmark, a deep watermarking framework for single-lead ECG. A fully convolutional generator embeds the payload without signal quantization, a detector decodes the message frame by frame and verifies watermark presence, and a frozen arrhythmia classifier penalizes changes that would alter diagnostic predictions. To bridge the gap between bit accuracy and whole-message recovery, payloads are protected by extended Hamming codes, validation-selected interleaving, and soft-decision decoding. On the MIT-BIH Arrhythmia Database, ECGWmark maintains over 90\% bit accuracy under time shifts of 10 to 50 samples, where transform baselines fall to chance level. With the coding layer, the protected 16-bit payload achieves 99.86\% exact recovery on clean segments and sustains over 90\% recovery under time shifts, 10\% sample deletion, and 10\% packet loss. Inter-patient evaluation confirms improved ventricular-beat sensitivity with preserved macro-F1. Requiring only 0.56 million parameters and about 15~ms per segment on a desktop CPU, ECGWmark highlights the need to assess physiological watermarking through whole-message recovery and diagnostic consistency.

0.18

ECGJev: Question-Conditioned Option Scoring for 12-Lead ECG Interpretation

cs.LG eess.SP ·2026-11-01 ·Zhuoyuan Li, Xidong Wu, Hanyu Cui, Jiansong Gao, Rongrong Fu +2 more
code/datahot direction similarity 0.62
Abstract · 5 pages (4 pages plus ethics statement and references), 2 figures. Submitted to IEEE ISBI 2027. Zhuoyuan Li and Xidong Wu contributed equall…

Clinical interpretation of a 12-lead electrocardiogram (ECG) often requires answering targeted diagnostic questions with clear, quantifiable decisions. We introduce ECGJev, the first ECG framework built on a decision decoder, a pretrained language model trained to return a probability for every listed option. A question-conditioned bridge compresses frozen waveform and spectrogram features into 16 ECG tokens, and a two-stage procedure adapts the bridge and low-rank adapters on the decoder's native option readout using diagnostic labels alone, scoring all options for a question in one forward pass. Evaluated across four ECG benchmarks, ECGJev achieves the highest macro-F1 on CPSC2018 (80.46%), PTB-XL (71.68%) and Chapman (96.86%), and generalizes competitively to the small-sample PTB cohort. Concept probes and perturbation analyses further show that its predictions rely directly on the ECG input, remain stable under option-order and wording variations, and transfer zero-shot to unseen waveform concepts. Our code is available at https://github.com/zhuoyuanli-lab/ECGJev.

0.34

Same path, different walk: a mechanistic comparison of looped and stacked transformer encoders on 12-lead ECG

cs.LG math.DS ·2026-09-14 ·Pawel Olszowiec, Michal Byra, Grzegorz Gruszczynski, Grzegorz Stefanski, Alberto Presta
similarity 0.61
Abstract · 5 pages, 7 figures

Recurrent Transformers reusing their weights rather than stacking $L$ distinct layers are becoming widely adopted due to their parameter efficiency [1,2,3]. However, the exact representational and dynamical differences between looped and stacked architectures remain uncharacterized. This paper presents a controlled study on the example of bViT model [1] applying one weight-tied block $L$ times. We train two models: bViT and standard ViT [4] on 12-lead electrocardiogram (ECG) classification tasks from the PTB-XL dataset under identical training protocols. Despite an $8.9\times$ parameter reduction, bViT achieves accuracy parity with ViT. Geometric similarity metrics demonstrate that both architectures construct comparable latent representations in an equivalent canonical order. Crucially, their dynamics differ: bViT exhibits smaller step sizes and inter-patient sensitivity, as well as near-neutral behavior away from the data manifold, whereas ViT exhibits collapsing dimensionality of representations and out-of-distribution feature expansion.

0.54

The Impact of Temporal Context Length and Encoding Strategies on Self-Supervised ECG Representation Learning

cs.LG eess.SP ·2026-08-13 ·Ahmed Sameh, Ramzi Al-Sharawi, Yogatheesan Varatharajah
publishedcode/data similarity 0.61
Abstract · Accepted at the 48th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2026). 6 pages, 2 figures…

Self-supervised electrocardiogram (ECG) models are often trained on a few seconds of ECG signal and, increasingly, on discretized token sequences. It remains unclear whether these choices sacrifice information needed for rhythm inference and longitudinal consistency in real-world ambulatory recordings. We present a controlled study on the Icentia11k single-lead dataset that varies (i) the input horizon (16 seconds, 1 minute, 5 minutes, and 10 minutes) and (ii) the front-end representation (continuous convolutional patch embeddings vs. fixed vector-quantized tokens), while holding the Transformer backbone and training protocol constant. Representations are assessed by downstream abnormal rhythm detection and by patient-level retrieval that probes cross-session stability. Our results show that increasing temporal context beyond 16-second snapshots yields stronger transfer and higher retrieval accuracy, with the strongest performance achieved by the 5- and 10-minute models, indicating improved capture of slow-varying rhythm dynamics and individual-specific structure. Across all evaluated horizons, continuous patch embeddings outperform discretized tokens, suggesting that quantization can discard clinically relevant waveform detail. These findings motivate ECG foundation models that emphasize extended context and continuous encoders for clinical prediction and similarity-based applications. Our code and pretrained models are publicly available at https://github.com/muha-0/ecg-ssl-representation-learning.

0.41

TransECG-SSL: Hybrid Self-Supervised Representation Learning for ECG Arrhythmia Classification

cs.LG ·2026-10-26 ·Ahtisham Ayyub, Fasih Haider, Muhammad Rehan Usman, Saturnino Luz, Christos Politis +1 more
similarity 0.59
Abstract · Submitted to IEEE Journal of Biomedical and Health Informatics

Cross-dataset generalisation remains a major challenge for automated electrocardiogram (ECG) arrhythmia classification because of differences in patient populations, acquisition protocols, annotation practices, and class distributions. Learning representations that remain informative across independent datasets could reduce reliance on dataset-specific labelled data. We propose TransECG-SSL, a self-supervised framework for transferable heartbeat representation learning from unlabelled ECG data. A Transformer encoder is pretrained using a hybrid objective combining masked signal reconstruction and contrastive learning, then frozen and evaluated by linear probing under patient-disjoint evaluation. Experiments across three public ECG databases and seven source--target settings show that Hybrid SSL consistently outperforms randomly initialised Transformers and achieves higher balanced accuracy than architecture-matched source-supervised pretraining. Reconstruction-only SSL remains competitive with Hybrid SSL, whereas contrastive-only pretraining performs substantially worse, highlighting the importance of masked reconstruction for transferability. Cross-dataset transfer nevertheless reduces balanced accuracy relative to target-specific self-supervised pretraining, while minority arrhythmia classes remain challenging under severe class imbalance. These findings demonstrate the potential of label-efficient self-supervised learning for cross-dataset ECG representation transfer while highlighting persistent limitations from domain shift and class imbalance.

0.24

ECGQuest: Benchmarking and Fine-Tuning Language Models for Electrocardiography

cs.CL cs.IR ·2026-08-31 ·Mohammadsina Hassannia, Matthew A. Reyna, Reza Sameni
benchmarkcode/data similarity 0.59
Abstract

Electrocardiogram (ECG) interpretation requires knowledge of cardiology, electrophysiology, clinical diagnosis, ECG waveforms, signal acquisition, and instrumentation. Existing language-model benchmarks, however, primarily assess broad medical knowledge or interpretation of individual ECG signals and images rather than the broader contextual knowledge required for ECG interpretation. We developed ECGQuest, a literature-grounded resource for evaluating and fine-tuning ECG-specific language models. A GPT-4o-based pipeline generated questions from 23 ECG references and Computing in Cardiology proceedings from 2003-2025. The final dataset contains 10,904 unique True/False questions paired with their negated forms (21,808 Q&A pairs). We evaluated three commercial and 20 open-source language models on a held-out test set in a zero-shot setting. Five open-source models with 7-14B parameters were fine-tuned using Low-Rank Adaptation, with BERT and BiomedBERT included as supervised encoder baselines. Generalization was assessed on ECG-related subsets of MedMCQA and MedQA converted to binary True/False questions using official answer keys. Zero-shot accuracy on ECGQuest ranged from 49.5% to 74.4%, with GPT-5 performing best. General-purpose models outperformed medically specialized models, several models showed strong True/False bias, and encoder baselines performed near chance. Fine-tuning improved all open-source models by 6.5-14.1%. Fine-tuned DeepSeek-R1-Distill-Qwen-14B reached 76.3% accuracy, while a five-model voting ensemble reached 78.5%. On MedMCQA and MedQA, fine-tuning mainly benefited weaker or class-biased models and did not consistently improve strong base models. ECGQuest provides a reproducible benchmark for contextual ECG knowledge and shows that parameter-efficient fine-tuning can make smaller language models competitive with substantially larger commercial models.

0.35