CS Paper Radar

Do Flow-Generation Tools Matter? Unsupervised Characterization of MQTT Traffic Across Public Datasets

Mohammed Rashed, Carlos García-Rubio, Celeste Campo

cs.NI cs.CR ·submitted 2026-10-21 14:05 UTC · v1

Abstract

Message Queue Telemetry Transport (MQTT) is widely used in IoT environments due to its lightweight publish-subscribe design and suitability for resource-constrained networks. While current MQTT security studies mainly focus on detecting attacks and optimizing classification performance, this work takes a different perspective by presenting a cross-dataset, tool-aware characterization of MQTT traffic at the flow level. This perspective is motivated by the fact that flow-generation is tool-dependent: flows are not directly observed in packet captures, but are constructed through tool-specific decisions about packet grouping, flow termination, and feature export. Using packet captures from 4 public MQTT datasets, we generated over 380K flows with 3 open-access flow-generation tools: Zeek-flowmeter, Argus, and NTLFlowLyzer. We analyzed the tool-specific flows using K-means, DBSCAN, and HDBSCAN, with and without PCA-based dimensionality reduction. We measured clustering quality using 2 metrics: (1) Silhouette score to assess label-independent cluster compactness and separation, and (2) Adjusted Rand Index to measure agreement with available labels. Our results show that the choice of flow-generation tool affects the resulting flow population, feature-space structure, and clustering behavior. Zeek-flowmeter achieves the best performance in several settings, particularly with K-means and PCA reduction to 10 components. Yet, no single tool consistently dominates across all datasets, algorithms, and evaluation metrics. These findings show that flow-level MQTT analysis should treat the dataset, flow-generation tool, preprocessing choices, clustering algorithm, and evaluation metric as interconnected experimental factors.

Comment
22 pages, 4 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Unmasking Shortcut Learning in IoT Intrusion Detection: A Forensic, Multi-Paradigm Evaluation of Feature Dependence and Data Leakage

cs.CR cs.LGcs.NI ·2026-09-23 ·Uday Shankar Roy, Mahbuba Jahan Minu
benchmark similarity 0.59
Abstract · Currently under review

Machine learning-based Network Intrusion Detection Systems often report near-perfect performance on IoT benchmarks. However, whether these models learn generalizable attack behavior or exploit spurious dataset shortcuts- such as static testbed IP/MAC addresses and chronological recording artifacts-remains an important question. We evaluate the CyberFlowIoT-GICAP benchmark, containing 3,617,388 flow records across 126 PCAP sessions with 849,395 benign flows. Four learning paradigms are evaluated across four feature configurations using PCAP-disjoint splits; LightGBM is additionally evaluated using conventional random-flow splitting. When only statistical flow behavior is used (Fbehav), LightGBM (92.58% +/- 8.18%), Random Forest (92.59% +/- 8.18%), and Deep MLP (92.55% +/- 8.18%) achieve nearly identical Macro-F1, indicating that performance is constrained by feature representation rather than model complexity. With raw timestamps (Ftstamp), tree-based models reach 99.28% Macro-F1, while the linear model remains at 90.62%, showing that nonlinear models can exploit dataset-specific temporal structure. Attack detectability is highly asymmetric: high-rate and active attacks maintain >99.8% recall from flow behavior alone in nonlinear models, whereas the DNS Beaconing drops from 27.78% to 0.00% recall when contextual features are removed. Conventional random-flow splitting increases attack recall by up to 14.00%, highlighting the effect of placing flows from the same sessions in both training and test sets. We conclude with a 4-point protocol checklist for realistic IoT NIDS evaluation.

0.13

Hierarchical Security Monitoring for Edge-IoT: A Formal Methods Approach

cs.CR cs.LOcs.SE ·2026-10-07 ·Nikolaos Kekatos, Marinelio Chintri, Panagiotis Katsaros, Alexios Lekidis, Tom Nianios +2 more
publishedhighly cited similarity 0.58
Abstract · 7 pages, 1 figure, 2 tables. Accepted at the 2026 IEEE International Conference on Cyber Security and Resilience (IEEE CSR 2026)

Cyber resiliency in edge-IoT deployments is fundamentally an economic problem: detection must keep critical processes operating under attack, but defender resources (compute, bandwidth, operator attention) are bounded. Centralised cloud monitoring offers expressive cross-device detection at prohibitive bandwidth cost; purely edge-local monitoring is cheap but blind to coordinated multi-device attacks where the asymmetric balance favours the attacker. We propose a lightweight hierarchical security-monitoring framework, built on formal runtime-verification methods, that occupies the practical middle ground at quantified cost. Each edge device runs a lightweight TeSSLa stream specification (size, payload validity, rate, and timestamp-drift predicates) that emits a four-valued verdict per aggregation window at sub-microsecond per-event cost; the gateway runs a parametric first-order MonPoly monitor over the per-device verdict streams at microsecond-scale per-verdict cost. The edge-to-gateway uplink carries roughly one Boolean per aggregation window per node, orders of magnitude smaller than the raw packet stream. The gateway tier detects coordinated attack patterns that no single-node monitor can see, shifting the asymmetric cost balance toward the defender. Every alert carries a witness set naming the device, the monitor tier, and the predicate that fired, providing an auditable record of the decision. We evaluate the framework on a container-host testbed spanning nominal and attacker nodes across four attack classes (buffer overflow, time spoofing, denial-of-service, and mixed advanced-persistent-threat patterns), and describe the edge- and gateway-tier specifications together with the cost-versus-coverage trade-off as monitor levels are added.

0.54

Adversarial RL for Port-Scan Evasion: Attacker Feature Visibility in Edge-Deployed IDS

cs.CR cs.LG ·2026-10-05 ·Logan Andrew North, Priya Sanjay Kaluskar, Shasi Kumar Ramachandran Prabhu, Peilong Li, Suman Saha
similarity 0.57
Abstract

Machine learning-based intrusion detection systems (IDS) are increasingly used in resource-constrained Internet of Things (IoT) environments, yet their robustness is often evaluated against static attacks rather than adversaries that adapt to detection feedback. This paper investigates adaptive port-scan evasion against ML-based IDS models deployed on a Raspberry Pi 3B+. We implement a live Zeek-based IDS pipeline with XGBoost, a multi-layer perceptron, and a 1D convolutional neural network trained on TON_IoT telemetry, and use a Deep Q-Network (DQN) adversary to learn evasive combinations of probe timing, TCP flags, and payload size under black-box, gray-box, and white-box feature-visibility settings. Although the deployed IDS models detect conventional port scans at 91.1--99.8%, DQN final-50-episode evasion rates range from 61.9% to 98.3% across feature-visibility settings. Greater feature visibility does not monotonically improve evasion, and its effect is model-dependent: against XGBoost, the black-box agent achieves 92.9% evasion, compared with 61.9% and 76.9% for gray-box and white-box agents, respectively, whereas 1D-CNN is most vulnerable under white-box access at 98.1%. Because standard DQN can overestimate action values, we additionally spot-check representative conditions using Double DQN. The gray-box condition remains unstable in this check, providing no evidence that overestimation bias alone explains the observed instability. These results show that limited feature knowledge can still enable effective adaptive evasion against static edge-deployed IDS models, motivating more robust defenses for IoT edge environments.

0.28

Making the Telemetry Pipeline Measurable: An Edge IoT Testbed and Dataset for End-to-End Latency, Provenance, and Energy-Aware Benchmarking

cs.DC cs.NIcs.SE ·2026-10-26 ·Dimitris Gkoulis
benchmark similarity 0.57
Abstract

Edge-centric IoT systems are increasingly evaluated through sensor values or platform features, yet the telemetry pipeline itself, the sequence of sensing, serialization, message mediation, ingestion, and persistence, is rarely released as a measurable object. This limits reproducible benchmarking of timeliness, provenance, and operational cost, especially in early-stage IoT design where real deployments and high-quality open data are often unavailable. I present a measurement-first edge IoT testbed and dataset that elevate timeliness, provenance, and energy to first-class telemetry dimensions. The testbed operationalizes a role-based event-driven architecture with explicit Source-Broker-Aggregator responsibilities and a schema-governed TelemetryMessage envelope (Avro) that embeds correlation identifiers and a uniform, cross-role vocabulary of stage-level timestamps. To trace mediation effects, I extend an MQTT broker (Eclipse Mosquitto) with a plugin that annotates messages with high-precision ingress/egress timestamps. I release the dataset in a dual representation: (i) replayable raw events as compressed Avro streams and (ii) analysis-ready long-format Parquet tables partitioned for efficient query and benchmarking, generated by a deterministic raw-to-tabular transformation pipeline. The contribution is an integrated engineering artifact and dataset built from established components. The demonstration describes one deployment through latency summaries, measurement-context distributions, and energy aggregates; it does not establish a new attribution method or a controlled performance benchmark. By co-releasing heterogeneous telemetry with pipeline timing and energy signals in both replayable and queryable forms, the artifact provides a basis for reproducible exploratory analysis and subsequent benchmarking studies.

0.20

QueueTrack: Framework for Detecting the Onset of Queueing in Mobile Networks

cs.NI ·2026-11-01 ·Hamid Hassani, Anudeep Karnam, Kishor Joshi, Sonia M. Heemstra, Ignas G. Niemegeers +1 more
published similarity 0.57
Abstract

In mobile networks, timely and accurate detection of the onset of queueing at path bottlenecks remains challenging for end-to-end congestion control algorithms (CCAs), where end hosts have limited visibility into queue status. Traditional congestion signals, such as packet loss, round-trip time (RTT), and one-way delay (OWD), have well-known limitations for early queue buildup detection. This paper presents QueueTrack, a practical end-host framework to detect the onset of queueing in mobile networks. QueueTrack computes a queue buildup indicator, denoted QBI, from endpoint-observable packet-level timing statistics, without requiring tight synchronization between end hosts or modifications to the existing network infrastructure. We develop and assess the complete QueueTrack machine learning (ML) pipeline, including data collection, labeling, model training, and testing, on a 5G testbed built on the OpenAirInterface platform, under a range of scenarios with competing TCP traffic. The experimental results show that QueueTrack can reliably detect the onset of queueing at the 5G base station using low-complexity classification models such as logistic regression. For example, QueueTrack achieves an F1 score of approximately 80% in a challenging mixed TCP/UDP scenario. These findings confirm that QueueTrack can serve as a practical framework for developing CCAs that target low delay, as well as for network telemetry and network monitoring in mobile networks.

0.26

A Feature-Rich Embedded NIDS with eBPF/XDP: Detector and Architecture Trade-offs

cs.CR cs.DCcs.NI ·2026-09-11 ·Shiqi Wu, Oleksii Koshovyi, Georgios Pseiridis Pseiras, Victor Morel, Romaric Duvignau
code/data similarity 0.57
Abstract

Distributed Denial-of-Service (DDoS) attacks remain a serious threat to transport networks, with recent attack volumes exceeding 30 Tbps, and the telecommunications industry being the main target. Recent work has yet to study the impact of the hosting software architecture on network monitoring solutions, or to assess recent algorithms for improving attack detection. This paper presents a Network Intrusion Detection System (NIDS) for DDoS detection in transport networks, developed in collaboration with Ericsson. Building on a statistical baseline, we improve detection effectiveness with an Isolation Forest trained on a wider set of flow features, extracted by GoFlowMeter, our open-source Go implementation of CICFlowMeter, and we integrate eBPF/XDP so that the NIDS filters real traffic at the kernel level. We further compare three deployments, monolithic, Kafka-based, and gRPC-based microservices, on a Raspberry Pi 5 testbed replaying the CIC-DDoS2019 dataset as real network traffic. Detection quality is governed mainly by the choice of detector rather than by the transport: the Isolation Forest raises recall and F1 score (0.965 live in the monolithic variant) over the baseline by flagging low-volume attack windows that the baseline misses. The transport is not neutral, however: gRPC reaches almost the same accuracy as the monolithic variant while adding less than 2 milliseconds of transport time per window, whereas the asynchronous Kafka pipeline trails by roughly nine percentage points and adds about 27 milliseconds. These findings clarify the trade-off between detection quality and architectural overhead when deploying a NIDS on resource-constrained hardware.

0.17

BotScan: An adaptive active probing approach for identifying live IoT Botnet C2 servers at scale

cs.NI ·2026-08-24 ·S M Maksudul Alam, Vivek Jain, Zhaowei Tan, Srikanth V. Krishnamurthy, Michalis Faloutsos
similarity 0.56
Abstract

How can we actively search and identify live C2 servers of botnets at scale? The scalability requirement introduces the need to utilize resources efficiently in terms of computation and number of probing packets. We propose BotScan, an approach for actively probing a large IP space to find the highest possible number of live C2 servers. The novelty of BotScan revolves around two insights, which we establish empirically. First, contrary to popular PC-centric observations, many modern IoT botnet communication protocols use packets with minimal customization, which we observe across six major families. Second, C2 servers exhibit exploitable behavioral patterns, such as strong spatial locality. We substantiate the first insight by developing a streamlined approach where, given malware binaries, we measure and taxonomize the "replayability" of its C2 communication protocol. Then, we introduce a behavior-adaptive probing strategy that: (a) exploits the spatial locality of C2 servers using a two-level segment-centric approach, and (b) adapts dynamically to the success of its probes. We validate the effectiveness of our method using 1,842 recently collected IoT binaries, and we explore a target space of 2.5M IP addresses. First, a replay-based method is applicable for at least 72% of the malware binaries. Second, our method outperforms baseline methods by finding approximately double the live C2 servers for the same number of probes. We also conduct two case-studies where we identify 896 live servers including 112 unreported C2 servers.

0.22

HiGAI-Net: Hierarchical Encrypted Traffic Classification for Generative AI Services

cs.NI ·2026-11-17 ·Madhan Raj Kanagarathinam, Jaheon Gu, Krishna M. Sivalingam, Kapal Dev
published similarity 0.56
Abstract · Paper accepted in IEEE Globecom 2026

The Radio Access Network (RAN) today does not have a reliable way to identify GenAI traffic (ChatGPT, Microsoft Copilot, or Google Gemini traffic) at the bearer level. TLS~1.3 moves SNI into the encrypted handshake record, and Encrypted ClientHello (ECH) removes it entirely from passive inspection; Gemini has partly shifted to QUIC where SNI visibility also depends on whether ECH is deployed. Without per-flow application identity, differentiated QoS for GenAI services is not possible. HiGAI-Net tackles this using only packet-header metadata, stacking three independently trained classifiers: a binary AI/Non-AI gate (98.5 % weighted F1), a per-service resolver across ChatGPT, Copilot, and Gemini (86.4 %), and a text-vs-voice/video mode classifier (61.0 %). The core innovation is a suite of sliding-window temporal features that track how DL/UL byte-ratios evolve within a session -- the per-quarter and per-half ratio trends, a trend differential $ΔR$, and a streaming regularity index. Tested on 44,492 labeled flows, gradient-boosted tree models consistently outperformed all deep-learning variants; a per-flow label cache brings amortised inference below 1~$μ$s per packet.

0.31