CS Paper Radar

IoMT-SecAlarmBench: A Counterfactual Benchmark for Integrity Attacks in IoMT

Emmanuel C. Ugwuabonyi, Dmitri Perkins

cs.CR ·submitted 2026-08-26 21:29 UTC · v1
benchmarkcode/dataunusual

Abstract

The Internet of Medical Things (IoMT) combines clinical physiological data with cyber-system information, creating challenges in determining whether an abnormal reading reflects a genuine physiological event, a device fault, or a cyber-attack within the expected physiological range. Answering this requires counterfactual ground truth, which no existing dataset provides. We present IoMT-SecAlarmBench, a semi-synthetic benchmark that injects controlled integrity attacks into genuine coupled ECG+PPG recordings using a structured experimental design combining four attack morphologies, four severity levels, two physiological plausibility conditions, and replay attacks. Each injected window retains its cause, attack subtype, and the clean signal that would have been observed without the attack. We evaluate six detectors from five method families using threshold-independent measures and a matched false-alarm budget. Results show no method consistently detects the most difficult cases: replay attacks and low-amplitude transient spikes remain close to chance-level performance across detectors. Results also reveal a trade-off between detecting attacks and distinguishing them from sensor faults: the best-performing detector on hard cases flags fault/artifact windows at 5.3 times its false-alarm rate on normal data. Three-way classification performs poorly for genuine physiological events, and a leakage audit of a dual-modality network dataset indicates previously reported IoMT intrusion-detection performance is partly driven by identifying information. Benchmark, generation code, preprocessing, evaluation tools, and datasheet are released.

Comment
10 pages, 2 figures, 5 tables. Submitted to IEEE International Conference on Big Data (BigData) 2026. Benchmark, generator, evaluation harness, and datasheet: https://github.com/techosystem/IoMT_SecAlarmBench

Closest papers

Nearest neighbours by meaning across the whole index.

IoTPwnBench: A Synthetic Benchmark for IoT Firmware Security Analysis

cs.CR cs.SE ·2026-11-11 ·Luca Ferrari, Marco Alecci, Luca Verderame, Jacques Klein, Francesco Pagano
benchmarkcode/data similarity 0.57
Abstract

Building and evaluating security tools for IoT firmware requires targets whose weaknesses are known in ad- vance and can be reproduced in a running device. Two problems remain: (1) firmware collected from real-world products provides realistic software, but hardware dependencies and undocumented defects make it difficult to establish what a security tool should be expected to find; and (2) existing benchmarks typically collect sets of vulnerable or misconfigured firmware images without providing purpose-built vulnerable services with explicit weaknesses that a security analysis tool can target. We present IoTPwnBench, a synthetic, modular benchmark that brings these requirements together. IoTPwnBench contains 716 vulnerable firmware images covering 86 weakness scenarios and 28 CWE classes across 11 device profiles, including industrial controllers, health monitors, and household devices. Each vulnerable image runs a plausible device application with simulated sensors and actuators that generate live readings over its native interfaces while exposing one target weakness mapped to a primary CWE. A documented exploitation payload and observable signal establish the expected behavior independently of the evaluated tool. Every vulnerable image is paired with a corrected twin that preserves the device and service while removing that weakness. Researchers can therefore compare the same target before and after correction to assess detection and false positives for the target weakness. The benchmark produces kernel and filesystem binaries for static inspection and dynamic testing under QEMU. Device profiles and weakness modules remain separate, allowing the same scenario to be reused across device contexts. Vulnerable targets were validated manually, and the open source build workflow regenerates the firmware bit-for-bit.

0.33

Demo: Matter-Based IoMT Telemetry and Security Analysis

cs.AR cs.ETcs.RO ·2026-11-12 ·Muhammad Hassan, Masooda Bashir, Carl Gunter
similarity 0.57
Abstract

Connected health devices increasingly support health monitoring outside clinical settings, making secure and accessible physiological data important for patients and caregivers. However, device-specific applications and interfaces complicate the integration of readings from different devices. The Matter standard provides standardized device commissioning and protected communication, but its cluster library does not represent the health attribute measurement. We present a preliminary demo study of a Matter-based IoMT testbed with a specific HealthVitals cluster. Implemented in the connectedhomeip SDK on a Raspberry~Pi~5, the cluster exposes simulated heart rate, blood oxygen saturation, and body temperature. We examine commissioning, attribute retrieval, communication under simulated packet loss, and operational traffic through a reference controller. The study demonstrates that this custom telemetry model can operate within the tested Matter configuration without modifying commissioning or secure-session mechanisms. The resulting testbed gives researchers and device developers a concrete platform for investigating health-oriented extensions. It also supports exploration of local telemetry integration that could help patients and caregivers access readings across connected devices.

0.23

Quantifying and Prioritizing Stealthy Source-Manipulation Threats Against Multi-Source Anomaly Detectors

cs.CR cs.LG ·2026-10-27 ·Indraveni Chebolu, Seema Panghal, Arnab Mallick, Ch A S Murty
publishedbenchmark similarity 0.56
Abstract · To be appeared at Workshop on AI for Cyber Threat Intelligence (WAITI) 2026 @ ACSAC 2026

Security pipelines often fuse several data sources and protect a downstream model with a cross-source anomaly detector. Such detectors are usually judged by their clean false-positive rate and by how well they catch random corruption. Neither metric captures an adversary who controls some sources and targets a specific downstream decision. We propose a threat-quantification score, the \emph{task-conditioned visibility margin}, defined as the least detector evidence a manipulation must reveal per unit of harm to the downstream task. The margin has a closed form, takes about $85\,μ$s per certificate, yields a minimum-evidence attack, and identifies task-relevant directions the detector cannot see. On a 17-channel industrial telemetry benchmark, a detector that flags 82.8\% of random single-channel corruptions is first-order blind on 64.7\% of task/subset pairs in an audit of one- to four-channel subsets. Task-conditioned manipulations cause more than twice as many falsely reassuring errors as generic stealthy ones, and more of them evade the alarm on four of five splits, mainly at small budgets. Used to prioritize audits on held-out subsets, the margin finds a median of four verified failures in ten checks, compared with none for random order and two for alarm-based ranking, and performs comparably to the task-gradient baseline. It also ranks vulnerable configurations of the learned GDN and TranAD detectors and of a five-source crop-yield panel.

0.36

Recording-Aware Evaluation of Multimodal OT Intrusion Detection

cs.CR cs.LG ·2026-10-16 ·Mustafa Umut Ozbek, Khalil El-Khatib, Li Yang
publishedbenchmarkcode/data similarity 0.56
Abstract · 12 pages, 3 figures, 5 tables. Accepted for presentation at the Industrial Control System Security Workshop (ICSS 2026), held in conjunction…

Machine-learning-based intrusion detection systems (IDSs) for operational technology (OT) are commonly evaluated on benchmark datasets in which normal and attack data are collected in separate recordings. In such settings, high performance may reflect recording-associated differences as well as attack behavior. We introduce a recording-aware evaluation procedure that compares attack windows with background from the same recording, uses prior data for model fitting and calibration, tests temporal alignment against randomized event schedules, and reports event coverage together with alarm burden. On the ProvICS multimodal dataset, cyber modalities strongly distinguish the benign and attack recordings before the first labeled attack. Same-recording evaluation reverses the apparent modality ranking across all studied window sizes: the physical view becomes the strongest individual modality, while cyber-only fusion shows the largest difference between cross-recording and same-recording AUC, reaching 0.54. Operational analysis further shows that permissive event-overlap metrics can overstate detection when a small number of long alarm episodes cover multiple actions. An external analysis on HAI shows smaller and reference-dependent recording sensitivity. These findings show that cross-recording benchmark performance should be interpreted alongside same-recording, forward, temporal, and operational evaluations.

0.44

Distinguishing Process Change, Instrument Fault and Data-Path Corruption in Streaming Telemetry: A Reference Implementation and Benchmark

eess.SP cs.LG ·2026-10-11 ·Oleksandr Klymenko
benchmarkcode/data similarity 0.56
Abstract · v2: typeset in LaTeX (IEEE format); references renumbered in order of citation; results unchanged. 8 pages, 2 figures, 7 tables. Code and da…

A reading departs from what a monitoring system expects for one of three reasons: the physical process changed, the instrument is failing, or the value was corrupted between the instrument and the software. The three call for different responses and look alike in a single sample. We state the three-cause problem, prove which pairs of causes one channel cannot separate, and propose a bank of online detectors - a Kalman tracker with CUSUM and EWMA charts on its innovation, a Hampel median with generalized-ESD confirmation, a robust noise statistic with ADWIN, exact-value checks and an optional two-channel complementary filter - combined by a deterministic, inspectable attribution rule. The detectors were merged into a public algorithm library. On 2,942 injected events of thirteen fault types the method detects 96.3% (95% interval 95.7-97.0) at 0.22 false alarms per hour and attributes 91.2% (90.6-91.8) of detected events correctly from one channel and 99.0% (98.6-99.4) with a second, rate channel; from one channel only 2 of 207 detected sensor drifts are attributed to the instrument, as the problem statement predicts. Injected into recorded industrial signals, the same catalogue is detected at 80.8% and attributed correctly at 84.5%, at 3.6 false alarms per hour. On the Skoltech Anomaly Benchmark the method detects 32 of 33 experiments with a median delay of 15 s, at 46 false alarms per hour against 11 for a rolling z-score.

0.58

Sensor-Free Shockable Rhythm Classification During CPR Using the Existing ECG Channel of Public-Access Automated External Defibrillators

eess.SP ·2026-11-19 ·Jaechan Lim, David Hicks, Matt Valentine, Ki H. Chon
similarity 0.55
Abstract

Automated external defibrillators (AEDs) are public-access devices intended for use by lay rescuers, yet they must pause chest compressions to analyze the rhythm because compressions corrupt the electrocardiogram (ECG). Removing this pause requires a method that uses the hardware already present in the device, fits within its processor budget, and meets the rhythm-recognition performance required for defibrillator approval. Existing approaches rely either on an auxiliary sensor, which entails a hardware redesign, or on an additional network that reconstructs the ECG before classification. This work classifies the compression-contaminated ECG directly, using only the existing ECG channel of the AED. A cascade of three compact convolutional networks trained with different class ratios verifies each decision with a second network biased toward the proposed outcome; disagreement yields an indeterminate outcome. Evaluated on an independent manufacturer-provided cohort of 464 subjects over 15 train-validation partitions, the cascade satisfies both adjudicable American Heart Association criteria under all six noise conditions at an eight-second window and a signal-to-artifact ratio of 0 dB, with sinus-rhythm specificity of 99.20-99.43% and other non-shockable specificity of 95.48-96.20%, leaving only 1.07-1.48% of segments indeterminate. The criteria are not met at -3 dB. Single networks matched in parameter count and inference cost do not reproduce this performance. Each decision evaluates two of the three networks and requires 6.68 ms on a single thread of a general-purpose processor, and eight-bit quantization reduces the model to 1.45 MiB without changing any compliance verdict. Thus, rhythm analysis during ongoing CPR can be implemented with the ECG hardware already present in public-access AEDs, without auxiliary sensing hardware, subject to validation on the target device.

0.17

Anytime-valid detection of LLM weight exfiltration

cs.CR stat.ME ·2026-10-08 ·Ines Ortega-Fernandez, Mateusz Kowalczyk, Keri Warr
published similarity 0.55
Abstract · Accepted for oral presentation at the NeurIPS 2026 workshop E-Values: From Statistics to ML (Paris, December 13 2026)

A compromised LLM inference server can leak model weights by encoding payload bits in otherwise plausible token choices. A replay of the same prompt in a trusted server can expose such deviations, but benign numerical nondeterminism also causes token mismatches. Patient attackers can therefore hide within normal variation unless evidence is combined across responses. We introduce a prompt-level e-process that calibrates whole-response mismatch events on trusted benign traffic and accumulates evidence sequentially while, under a calibration-transfer assumption, controlling the probability of any false alarm over an unbounded monitoring horizon. We evaluate it on four models against a seed-blind attack and a stronger seed-aware attack that hides payload bits only in near-ties to remain stealthy, analyzing the channel capacity vs detectability trade-off. Compared with a hard per-token alarm, the e-process combines weak evidence across responses while providing explicit anytime false-alarm control.

0.38

Uncertainty-Guided Counterfactual Sensor Selection and Safety-Gated Recognition for Autonomous Sensing Agents in IoT Systems

cs.NI ·2026-10-26 ·Shirong Guo, Hongming Zhou, Muhammad Attique Khan, Jinfeng Xu, Yonis Gulzar +2 more
similarity 0.53
Abstract · 15 pages, 13 figures

Autonomous IoT systems rely on distributed sensing, but sensor faults and communication loss can leave multichannel evidence incomplete or unreliable. Their agents must therefore classify measurements, assess evidence sufficiency, select a sensor to reacquire, and hand off unsafe decisions. We formulate counterfactual sensing: a degraded window defines the factual state, each candidate restoration defines a controlled alternative, and a pre-request utility model estimates loss reduction. Physics-Guided Invariant-Kinematic Fusion (PGIK-Fusion) constructs class evidence, while calibrated risk and cost-sensitive utility select acceptance, sensor request, or handoff from factual information. On the Logistic Activity Recognition Challenge dataset, PGIK-Fusion achieves 0.768 accuracy and a macro-averaged F1 score (Macro-F1) of 0.696, exceeding extreme gradient boosting (XGBoost)--one-dimensional convolutional neural network (1D-CNN) fusion by 2.49 and 2.00 percentage points, respectively. Across seven predefined degradation conditions on held-out subjects, the policy reduces unsafe acceptance from 33.0% to 13.8%; relative to an action-rate-matched random policy, it raises automation accuracy by 18.5 percentage points and lowers expected cost by 24.7%. In the auxiliary within-event wind-turbine study, PGIK-Fusion reaches 0.975 Macro-F1, improves on XGBoost--1D-CNN by 1.17 percentage points, and reduces unsafe acceptance from 20.1% to 8.0%. The evaluation characterizes the controlled responses of a frozen recognition model to specified sensor restorations under predefined fault families and severities. Within this scope, the results demonstrate that calibrated recognition, targeted evidence acquisition, and conservative deferral can form a unified perception--action loop that reduces unsafe automation under sensor degradation.

0.28