CS Paper Radar

Beyond Detection Accuracy: Measuring Explanation Cost, Stability, and Utility for Resource-Aware IoT Intrusion Detection

Abdurrahman Tolay

cs.CR cs.LGcs.NI ·submitted 2026-08-11 01:18 UTC · v1

Abstract

Machine-learning intrusion-detection studies commonly emphasize predictive accuracy while treating explanation generation as a computationally free post-processing step. This study jointly evaluates predictive effectiveness, explanation cost, local explanation stability, and selective explanation for binary Internet of Things (IoT) intrusion detection. A leakage-safe CICIoT2023 corpus was constructed using exact 39-feature hashes, non-finite-value handling, exact-feature deduplication, conservative label-collision removal, and deterministic hash-level partitioning. Logistic Regression, Decision Tree, Random Forest, and XGBoost were evaluated on natural and balanced test distributions. TreeSHAP cost was measured, stability was assessed under prediction-preserving perturbations, and validation-calibrated policies were used to allocate explanation workload. XGBoost provided the strongest overall predictive profile, while Random Forest produced the lowest false-positive rate. At 5,000 samples, TreeSHAP required 700.759 s for Random Forest and 1.471 s for XGBoost. Random Forest showed the strongest overall base-level explanation stability; XGBoost retained high rank and directional consistency but showed greater top-feature turnover and attribution-magnitude drift. On the balanced test, about 90% false-negative explanation coverage permitted 28-32% compute savings, while about 95% coverage permitted 15-23% savings. Savings were much smaller under the attack-heavy natural prevalence. These results show that operationally useful explainable IoT intrusion detection depends on predictive quality, explanation cost, local stability, workload prevalence, and selective invocation rather than detection accuracy alone.

Comment
43 pages, 3 figures, 15 tables. Submitted to Internet of Things. Reproducibility materials are publicly archived on Zenodo

Closest papers

Nearest neighbours by meaning across the whole index.

Towards Lightweight IoT Intrusion Detection: Feature Optimization and Explainable Machine Learning for Resource-Constrained Environment

cs.CR ·2026-11-07 ·Shaker Nawasra, Munther Abualkibash
published similarity 0.77
Abstract

Machine learning-based intrusion detection systems (IDSs) can improve IoT security but often rely on large feature sets that increase computational overhead and limit deployment on resource-constrained devices. This paper presents a resource-efficient and explainable feature optimization framework for lightweight IoT intrusion detection using the CICIoT2023 dataset. A Random Forest model was used to rank feature importance and identify compact feature subsets. Experimental results show that a three-feature model achieved an MCC of 0.9622 while reducing the feature space by 92.1%, whereas a six feature model achieved an MCC of 0.9675 with an 84.2% reduction. Furthermore, a fifteen-feature model achieved an MCC of 0.9744, preserving 99.97% of the performance of the full 38-feature baseline model while reducing the feature space by 60.5%. SHAP-based explainability analysis validated the selected features and provided interpretable insights into model decisions. The results demonstrate that lightweight, explainable, and high-performance IoT intrusion detection can be achieved using only a small subset of carefully selected features, significantly reducing resource requirements with minimal loss of detection effectiveness.

0.20

Unmasking Shortcut Learning in IoT Intrusion Detection: A Forensic, Multi-Paradigm Evaluation of Feature Dependence and Data Leakage

cs.CR cs.LGcs.NI ·2026-09-23 ·Uday Shankar Roy, Mahbuba Jahan Minu
benchmark similarity 0.69
Abstract · Currently under review

Machine learning-based Network Intrusion Detection Systems often report near-perfect performance on IoT benchmarks. However, whether these models learn generalizable attack behavior or exploit spurious dataset shortcuts- such as static testbed IP/MAC addresses and chronological recording artifacts-remains an important question. We evaluate the CyberFlowIoT-GICAP benchmark, containing 3,617,388 flow records across 126 PCAP sessions with 849,395 benign flows. Four learning paradigms are evaluated across four feature configurations using PCAP-disjoint splits; LightGBM is additionally evaluated using conventional random-flow splitting. When only statistical flow behavior is used (Fbehav), LightGBM (92.58% +/- 8.18%), Random Forest (92.59% +/- 8.18%), and Deep MLP (92.55% +/- 8.18%) achieve nearly identical Macro-F1, indicating that performance is constrained by feature representation rather than model complexity. With raw timestamps (Ftstamp), tree-based models reach 99.28% Macro-F1, while the linear model remains at 90.62%, showing that nonlinear models can exploit dataset-specific temporal structure. Attack detectability is highly asymmetric: high-rate and active attacks maintain >99.8% recall from flow behavior alone in nonlinear models, whereas the DNS Beaconing drops from 27.78% to 0.00% recall when contextual features are removed. Conventional random-flow splitting increases attack recall by up to 14.00%, highlighting the effect of placing flows from the same sessions in both training and test sets. We conclude with a 4-point protocol checklist for realistic IoT NIDS evaluation.

0.33

Evasion Attacks on Cost-Utility-Based Adversarial Training for Online AutoML in IoT Networks

cs.CR cs.LG ·2026-09-25 ·Chukwunonso Henry Nwokoye, Wajiha Zaheer, Khalil El-Khatib, Li Yang
publishedcode/data similarity 0.64
Abstract · Accepted to the IEEE Middle East Conference on Communications and Networking (MECOM 2026). Code is available at: https://github.com/ChiNonso…

As Internet of Things (IoT) networks increasingly depend on machine learning for anomaly, malware, intrusion detection, and network monitoring, such systems have become attractive targets for evasion attacks. Evasion attacks pose a major security risk because an adversary intentionally modifies input data to mislead a trained model into producing incorrect predictions while evading detection. This study evaluates the impact of black-box evasion attacks on a cost-utility-based adversarial training defense strategy in an Online AutoML context for IoT networks. Specifically, evasion attacks were applied to online learners, including Hoeffding Tree (HT), Leveraging Bagging (LB), Streaming Random Patches (SRP), Hoeffding Adaptive Tree (HAT), and Adaptive Random Forest (ARF). By developing naive and adversarially trained (AT) versions of these online learners, we generated clean and adversarial accuracies for each model. The results show that the AT versions of LB and SRP performed best, achieving the highest adversarial accuracy (0.985) and high clean accuracy (0.993) at the highest cost budget of 1.00, with a maximum accuracy reduction of only 0.8%. Finally, drift detection was conducted using the Early Drift Detection Method (EDDM).

0.42

Provenance, Not Behaviour: A Serialisation Artifact in Edge-IIoTset and a Leakage-Free Benchmark for Precision-Agriculture Intrusion Detection

cs.CR cs.LG ·2026-08-16 ·Mostafa M. Galal
benchmark similarity 0.64
Abstract · 29 pages, 13 tables. Code: https://doi.org/10.5281/zenodo.21941210 Benchmark: https://doi.org/10.5281/zenodo.21941319

Edge-IIoTset is the reference benchmark for machine-learning intrusion detection in the industrial Internet of Things, and results reported on it cluster above 99%. We show that much of that performance is not intrusion detection. The preprocessing recipe distributed with the dataset instructs researchers to one-hot encode seven categorical columns. Four of them separate attack from normal traffic with an accuracy of 1.0000 on their own, through the spelling of the placeholder written for an absent protocol field: the string "0" in the normal-traffic branch of the dataset build against "0.0" in the attack branch. The label is recoverable from a serialisation artifact encoding file provenance, with no network behaviour modelled, and separates every row of both curated subsets. Under 5-fold x 3-repeat cross-validation, five of six standard classifiers attain exactly 1.0000 +/- 0.0000 accuracy and the sixth attains 0.99998. Under a corrected protocol, naive Bayes falls by 0.3005 macro-F1 and the strongest model settles at 0.9503 +/- 0.0011. Label, ordinal and frequency encoding leak identically. Because the curated subsets also lack Modbus and per-device identity, we rebuild the benchmark from the raw captures under uniform parsing, producing AgriEdge: 1,276,122 rows, five devices with full attribution, and no column separating the classes above 0.0288. A leave-one-device-out sweep locates the generalisation boundary at the perception/actuation layer, where random forest falls from 0.9988 to 0.5083 balanced accuracy. Non-IID federated partitioning costs at most 0.0037 macro-F1, but a 20-round LoRaWAN training run costs 4.6 hours of uplink.

0.22

Passive Hybrid Network-Based Intrusion Detection System (Hybrid-NIDS) Combining Suricata and Random Forest

cs.CR ·2026-09-21 ·Quoc-Cuong Tang, Hoang-Lam Huynh, Van-Tri Phan, Khuong Nguyen-An
publishedbenchmark similarity 0.63
Abstract · Accepted for publication in the Proceedings of the 29th National Conference on Selected Issues of Information and Communication Technology (…

This paper evaluates a passive Hybrid Network-based Intrusion Detection System (Hybrid-NIDS) prototype that combines Suricata with Random Forest flow classification and centralized ELK-based alert handling. The study explicitly separates benchmark evaluation from PCAP/live operational validation and controls exact feature-duplicate leakage using feature hashing and group-aware splitting. From 2,540,047 UNSW-NB15 records, 453 conflicting-label hash groups containing 1,879 rows were removed; the resulting Development and Hold-out sets have zero exact feature-hash overlap. RF-41 achieved F1 = 0.971360 and ROC-AUC = 0.999671, while the NFStream-compatible RF-21 achieved F1 = 0.970148 on the same prepared hold-out boundary. However, operational validation revealed substantial benchmark-to-deployment domain shift: on a labeled laboratory PCAP, RF-21 and the strictly correlated branch achieved recall of only 0.0095, and RF-21 produced no alerts in five additional 60-second attack sessions. An unlabeled normal-traffic test produced 439 alerts from 2,375 flows; this value is reported only as an alert ratio and is not interpreted as a false-positive rate. These results show that strong performance on a public benchmark does not directly translate into operational effectiveness. Accordingly, the current Hybrid-NIDS should be interpreted as a passive prototype and evaluation framework, and the reported experiments do not demonstrate that Suricata-Random Forest correlation provides better operational detection than Suricata alone.

0.38

Empirical Evaluation of Machine Learning for Cloud Cryptojacking Detection: Effects of Preprocessing and Evaluation Protocols

cs.CR ·2026-10-14 ·Amitabh Chakravorty, Nelly Elsayed
similarity 0.62
Abstract

Artificial intelligence (AI) has been widely applied to cloud cryptojacking detection. Classical machine learning models remain among the most widely used approaches for cloud cryptojacking detection. This paper presents an empirical evaluation of six widely used classical machine learning models for AI-based cloud cryptojacking detection under different preprocessing and evaluation settings. Because no public cloud-native cryptojacking dataset is currently available, experiments were conducted using two publicly available proxy datasets, DS2OS and NSL-KDD. The evaluated models include Random Forest, XGBoost, LightGBM, Gradient Boosting, Decision Tree, and K-Nearest Neighbors (KNN). Experimental results show that preprocessing choices and evaluation protocols have a measurable impact on model performance. On DS2OS, removing identifier-related features reduced the accuracy of most models to approximately 96%, while KNN achieved higher accuracy with lower attack recall. On NSL-KDD, models obtained 77 to 80\% accuracy on the official test set containing unseen attack types, compared with above 99.5% under a random train-test split. Attack-class precision, recall, and Matthews Correlation Coefficient (MCC) further provide complementary insights into classifier performance beyond overall accuracy. The results provide an empirical comparison of commonly used classical machine learning models under consistent experimental settings and offer insights into how preprocessing, dataset selection, and evaluation methodology influence reported performance.

0.21

Online AutoML: Evaluating Poisoning Attacks on Cost-Utility Adversarial Training Defence Strategy for IoT Networks

cs.CR cs.LG ·2026-10-05 ·Chukwunonso Henry Nwokoye, Khalil El-Khatib, Li Yang
publishedcode/data similarity 0.62
Abstract · Accepted and to appear in IEEE CASCON 2026. Code is available at: https://github.com/ChiNonsoHenry16/Label-Flip-NoiseInjection_OnAutoML

Machine learning (ML)-powered poisoning attack vectors are adversarial maneuvers whereby an attacker intentionally inserts, corrupts, or alters training data to distort an ML model's learning process. The objective is to diminish model efficacy, instill biases, induce misclassifications, or include concealed backdoors that may be attacked during implementation. In streaming contexts, poisoning attacks pose significant risks since models perpetually update based on incoming streams of data. An assailant may incrementally introduce harmful samples into this data stream, leading the model to assimilate erroneous features over time without timely identification. Therefore, this study is aimed at evaluating the efficacy of the adversarial training (AT) defense approach against poisoning attacks (label flip and noise injection) using an online AutoML pipeline for Internet of Things (IoT) networks. Specifically, poisoning attacks (label flip and noise injection) were applied to streaming-capable AutoML learners (Hoeffding Tree (HT), Leveraging Bagging (LB), Adaptive Random Forest (ARF), Hoeffding Adaptive Tree (HAT), and Streaming Random Patches (SRP)). Under the strongest poisoning rate (PR = 1.0), AT-SRP achieved the highest F1-score against label flip poisoning (0.904), while AT-LB achieved the highest F1-score against noise-injection poisoning (0.933). Finally, several drift detection methods were used for rolling accuracy and prequential evaluation.

0.27

Adversarial Robustness Is Not Zero-Day Generalization: A Controlled Evaluation of Flow-Based Intrusion Detectors on Corrected CIC-IDS2017

cs.CR cs.LG ·2026-10-31 ·Abdulhamid Zaro
code/data similarity 0.62
Abstract · 10 pages, 10 tables. Code and per-seed results: https://github.com/abdulhamid-zaro/robustness-vs-zeroday-nids

Synthetic variants of known attacks, including feature-space adversarial perturbations, are sometimes used as stand-ins for unseen ("zero-day") attacks when evaluating machine-learning network intrusion detection systems (NIDS). We test whether this proxy holds. On the corrected CIC-IDS2017 release we train Random Forest, XGBoost, a 1D-CNN and an FT-Transformer and compare, at a fixed 1% false-positive rate over five seeds, detection under constrained perturbations of known attacks (gradient-based and black-box attacks on 42 attacker-controllable features) with leave-one-attack-family-out (LOAO) detection, where an entire family is absent from training. Detection under the strongest perturbation we tested, with budgets of at most 0.2, stays between 90.9% and 99.7%, yet mean detection of held-out families falls to 76.2% (RF), 40.2% (XGBoost), 25.8% (1D-CNN) and 22.8% (FT-Transformer). Robustness exceeds LOAO detection for every model (gaps of 24 to 72 percentage points, 23 to 66 when robustness is also averaged over families; Holm-adjusted p $\le$ 0.00096), and the two measures agree at most on a coarse ranking of the models. An exact mixed-integer evasion search confirms the tree ensembles' scores. Benign-only anomaly detectors and a simple multi-class open-set baseline do not close the gap. A temporal split on 5 seeds (3 of the 4 models) leaves the gap in place. On CSE-CIC-IDS2018 the gap recurs for every model (16 to 68 points). On these benchmarks and under this threat model, a robustness score should not be read as a zero-day score, and the two should be reported separately. Code and per-seed results are publicly available.

0.40