CS Paper Radar

Passive Hybrid Network-Based Intrusion Detection System (Hybrid-NIDS) Combining Suricata and Random Forest

Quoc-Cuong Tang, Hoang-Lam Huynh, Van-Tri Phan, Khuong Nguyen-An

cs.CR ·submitted 2026-09-21 10:40 UTC · v1
publishedbenchmark

Abstract

This paper evaluates a passive Hybrid Network-based Intrusion Detection System (Hybrid-NIDS) prototype that combines Suricata with Random Forest flow classification and centralized ELK-based alert handling. The study explicitly separates benchmark evaluation from PCAP/live operational validation and controls exact feature-duplicate leakage using feature hashing and group-aware splitting. From 2,540,047 UNSW-NB15 records, 453 conflicting-label hash groups containing 1,879 rows were removed; the resulting Development and Hold-out sets have zero exact feature-hash overlap. RF-41 achieved F1 = 0.971360 and ROC-AUC = 0.999671, while the NFStream-compatible RF-21 achieved F1 = 0.970148 on the same prepared hold-out boundary. However, operational validation revealed substantial benchmark-to-deployment domain shift: on a labeled laboratory PCAP, RF-21 and the strictly correlated branch achieved recall of only 0.0095, and RF-21 produced no alerts in five additional 60-second attack sessions. An unlabeled normal-traffic test produced 439 alerts from 2,375 flows; this value is reported only as an alert ratio and is not interpreted as a false-positive rate. These results show that strong performance on a public benchmark does not directly translate into operational effectiveness. Accordingly, the current Hybrid-NIDS should be interpreted as a passive prototype and evaluation framework, and the reported experiments do not demonstrate that Suricata-Random Forest correlation provides better operational detection than Suricata alone.

Comment
Accepted for publication in the Proceedings of the 29th National Conference on Selected Issues of Information and Communication Technology (VNICT 2026), Hanoi, Vietnam, November 7-8, 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Candidate Comparability Before Promotion: Conditional Validation in Adaptive Network Intrusion Detection

cs.CR cs.LG ·2026-09-03 ·Roberto Fernández-Barrios, Iker Pastor-López, Amaia Pikatza-Huerga, Pablo García Bringas
benchmark similarity 0.65
Abstract · 29 pages, 3 figures, 11 tables. Supplementary material (Online Resource 1) is included as an ancillary file

Adaptive network intrusion detection systems retrain classifiers after drift alarms, but an alarm detects change; it does not establish that a challenger should replace the deployed incumbent. Promotion is security-relevant because it changes the model responsible for subsequent attack detection, and evaluating it has a methodological problem: promotion conclusions may depend on how the challenger was constructed and on how much evidence supports it. We test that dependence on CICIDS2017, UNSW-NB15 and ToN-IoT with self-contained challenger pipelines, nested candidate-size controls, a common-harness comparison of nine update policies, and a final sensitivity confining every exact feature vector to one evaluation, training or probe role. Incumbent-owned frozen preprocessing amplified apparent promotion harm; with self-contained challenger pipelines the mean full-drift harm did not persist. Raising nominal candidate evidence from 512 to 2,000 samples per class improved promotion under pool-constructed progressive drift by +0.53, +1.67 and +0.38 balanced-accuracy points: positive and statistically resolved in all three benchmarks, but materially benchmark-dependent rather than homogeneous, and driven mainly by fewer false positives. Policy conclusions were partially robust: policy ordering changed with candidate comparability, no policy globally dominated, and earlier compatibility statements for a label-free estimator and a calibrated ensemble narrowed. Validation helped evidence-disadvantaged challengers but added no average benefit at parity. Thirteen replays on real, time-ordered traffic showed no net harm from always deploying. Challenger construction and evidence should be controlled, reported and interpreted explicitly when promotion is evaluated.

0.39

Beyond Detection Accuracy: Measuring Explanation Cost, Stability, and Utility for Resource-Aware IoT Intrusion Detection

cs.CR cs.LGcs.NI ·2026-08-11 ·Abdurrahman Tolay
similarity 0.63
Abstract · 43 pages, 3 figures, 15 tables. Submitted to Internet of Things. Reproducibility materials are publicly archived on Zenodo

Machine-learning intrusion-detection studies commonly emphasize predictive accuracy while treating explanation generation as a computationally free post-processing step. This study jointly evaluates predictive effectiveness, explanation cost, local explanation stability, and selective explanation for binary Internet of Things (IoT) intrusion detection. A leakage-safe CICIoT2023 corpus was constructed using exact 39-feature hashes, non-finite-value handling, exact-feature deduplication, conservative label-collision removal, and deterministic hash-level partitioning. Logistic Regression, Decision Tree, Random Forest, and XGBoost were evaluated on natural and balanced test distributions. TreeSHAP cost was measured, stability was assessed under prediction-preserving perturbations, and validation-calibrated policies were used to allocate explanation workload. XGBoost provided the strongest overall predictive profile, while Random Forest produced the lowest false-positive rate. At 5,000 samples, TreeSHAP required 700.759 s for Random Forest and 1.471 s for XGBoost. Random Forest showed the strongest overall base-level explanation stability; XGBoost retained high rank and directional consistency but showed greater top-feature turnover and attribution-magnitude drift. On the balanced test, about 90% false-negative explanation coverage permitted 28-32% compute savings, while about 95% coverage permitted 15-23% savings. Savings were much smaller under the attack-heavy natural prevalence. These results show that operationally useful explainable IoT intrusion detection depends on predictive quality, explanation cost, local stability, workload prevalence, and selective invocation rather than detection accuracy alone.

0.12

Multi-Class, Multi-Tier Network Intrusion Detection: A Comprehensive and Reproducible Benchmark

cs.CR cs.LG ·2026-09-28 ·Yufeng Xin, Bryant Goseland, Mohamed Rahouti
benchmarkcode/data similarity 0.62
Abstract

Machine learning (ML) and deep learning (DL) have dominated Intrusion Detection System (IDS) research in recent years. Unfortunately, many existing studies have produced inflated results and unreliable benchmarks due to critical oversights and mistakes in the ML and DL pipeline, from data collection and labeling to feature engineering and model training and evaluation. CIC-IDS2017 is a standard benchmark for network intrusion detection. Still, many published results on this dataset are difficult to compare due to labeling errors, inconsistent flow extraction, potential leakage, and performance evaluation metrics dominated by benign traffic. In this paper, we present a comprehensive benchmark with corrected PCAP-level labeling and a complete evaluation pipeline with diverse ML models. We evaluate eleven tabular classifiers at three nested levels: binary attack detection, nine-class attack-family attribution, and fifteen-class fine-grained classification. A soft-voting ensemble of Random Forest, XGBoost, and LightGBM obtains the best fine-tier macro-F1 of 0.955, with coarse and binary macro-F1 scores of 0.980 and 0.999, respectively. We further conducted a feature selection study based on an analysis of feature importance. This comprehensive benchmark pipeline is configurable and open-source, enabling new feature extraction and model plugins for new datasets. Future work should use this pipeline as a reference point for richer features, rare-class analysis, and model generalization towards new datasets and attack classes.

0.18

Jev-IDS: System One Models for Network Intrusion Detection

cs.CR cs.AI ·2026-10-01 ·Paulo Severo, Silvio E. Quincozes, Amanda Dias
similarity 0.62
Abstract

Machine-learning Network Intrusion Detection Systems (IDS) depend on substantial labeled datasets and task-specific training, whereas Large Language Models (LLMs) detection can analyze flow records directly but incurs higher inference cost and latency, with less constrained outputs. This paper presents JEV-IDS, an open experimental general NIDS based on the Jev System One Model (SOM) to detect zero day intrusions Under label scarcity. JEV-IDS serializes one flow per request and asks JEV two questions: a binary attack probability and a finite-choice traffic category. Our results show that, at k=1, JEV was 4.8 times faster and 3.8 times cheaper than GPT-5.6 Luna, with 1.5 times higher novel-attack recall; it also produced 15 times fewer false alarms than a low-data Random Forest. Across 5,400 decisions on a 300-flow NSL-KDD pilot split, JEV achieved F1-Score 0.859, precision 0.941, recall 0.790, and novel-attack recall 0.838. Increasing k to 2 reduced its F1-Score to 0.839.

0.22

Adversarial Robustness Is Not Zero-Day Generalization: A Controlled Evaluation of Flow-Based Intrusion Detectors on Corrected CIC-IDS2017

cs.CR cs.LG ·2026-10-31 ·Abdulhamid Zaro
code/data similarity 0.61
Abstract · 10 pages, 10 tables. Code and per-seed results: https://github.com/abdulhamid-zaro/robustness-vs-zeroday-nids

Synthetic variants of known attacks, including feature-space adversarial perturbations, are sometimes used as stand-ins for unseen ("zero-day") attacks when evaluating machine-learning network intrusion detection systems (NIDS). We test whether this proxy holds. On the corrected CIC-IDS2017 release we train Random Forest, XGBoost, a 1D-CNN and an FT-Transformer and compare, at a fixed 1% false-positive rate over five seeds, detection under constrained perturbations of known attacks (gradient-based and black-box attacks on 42 attacker-controllable features) with leave-one-attack-family-out (LOAO) detection, where an entire family is absent from training. Detection under the strongest perturbation we tested, with budgets of at most 0.2, stays between 90.9% and 99.7%, yet mean detection of held-out families falls to 76.2% (RF), 40.2% (XGBoost), 25.8% (1D-CNN) and 22.8% (FT-Transformer). Robustness exceeds LOAO detection for every model (gaps of 24 to 72 percentage points, 23 to 66 when robustness is also averaged over families; Holm-adjusted p $\le$ 0.00096), and the two measures agree at most on a coarse ranking of the models. An exact mixed-integer evasion search confirms the tree ensembles' scores. Benign-only anomaly detectors and a simple multi-class open-set baseline do not close the gap. A temporal split on 5 seeds (3 of the 4 models) leaves the gap in place. On CSE-CIC-IDS2018 the gap recurs for every model (16 to 68 points). On these benchmarks and under this threat model, a robustness score should not be read as a zero-day score, and the two should be reported separately. Code and per-seed results are publicly available.

0.40

Provenance, Not Behaviour: A Serialisation Artifact in Edge-IIoTset and a Leakage-Free Benchmark for Precision-Agriculture Intrusion Detection

cs.CR cs.LG ·2026-08-16 ·Mostafa M. Galal
benchmark similarity 0.60
Abstract · 29 pages, 13 tables. Code: https://doi.org/10.5281/zenodo.21941210 Benchmark: https://doi.org/10.5281/zenodo.21941319

Edge-IIoTset is the reference benchmark for machine-learning intrusion detection in the industrial Internet of Things, and results reported on it cluster above 99%. We show that much of that performance is not intrusion detection. The preprocessing recipe distributed with the dataset instructs researchers to one-hot encode seven categorical columns. Four of them separate attack from normal traffic with an accuracy of 1.0000 on their own, through the spelling of the placeholder written for an absent protocol field: the string "0" in the normal-traffic branch of the dataset build against "0.0" in the attack branch. The label is recoverable from a serialisation artifact encoding file provenance, with no network behaviour modelled, and separates every row of both curated subsets. Under 5-fold x 3-repeat cross-validation, five of six standard classifiers attain exactly 1.0000 +/- 0.0000 accuracy and the sixth attains 0.99998. Under a corrected protocol, naive Bayes falls by 0.3005 macro-F1 and the strongest model settles at 0.9503 +/- 0.0011. Label, ordinal and frequency encoding leak identically. Because the curated subsets also lack Modbus and per-device identity, we rebuild the benchmark from the raw captures under uniform parsing, producing AgriEdge: 1,276,122 rows, five devices with full attribution, and no column separating the classes above 0.0288. A leave-one-device-out sweep locates the generalisation boundary at the perception/actuation layer, where random forest falls from 0.9988 to 0.5083 balanced accuracy. Non-IID federated partitioning costs at most 0.0037 macro-F1, but a 20-round LoRaWAN training run costs 4.6 hours of uplink.

0.22

Unmasking Shortcut Learning in IoT Intrusion Detection: A Forensic, Multi-Paradigm Evaluation of Feature Dependence and Data Leakage

cs.CR cs.LGcs.NI ·2026-09-23 ·Uday Shankar Roy, Mahbuba Jahan Minu
benchmark similarity 0.60
Abstract · Currently under review

Machine learning-based Network Intrusion Detection Systems often report near-perfect performance on IoT benchmarks. However, whether these models learn generalizable attack behavior or exploit spurious dataset shortcuts- such as static testbed IP/MAC addresses and chronological recording artifacts-remains an important question. We evaluate the CyberFlowIoT-GICAP benchmark, containing 3,617,388 flow records across 126 PCAP sessions with 849,395 benign flows. Four learning paradigms are evaluated across four feature configurations using PCAP-disjoint splits; LightGBM is additionally evaluated using conventional random-flow splitting. When only statistical flow behavior is used (Fbehav), LightGBM (92.58% +/- 8.18%), Random Forest (92.59% +/- 8.18%), and Deep MLP (92.55% +/- 8.18%) achieve nearly identical Macro-F1, indicating that performance is constrained by feature representation rather than model complexity. With raw timestamps (Ftstamp), tree-based models reach 99.28% Macro-F1, while the linear model remains at 90.62%, showing that nonlinear models can exploit dataset-specific temporal structure. Attack detectability is highly asymmetric: high-rate and active attacks maintain >99.8% recall from flow behavior alone in nonlinear models, whereas the DNS Beaconing drops from 27.78% to 0.00% recall when contextual features are removed. Conventional random-flow splitting increases attack recall by up to 14.00%, highlighting the effect of placing flows from the same sessions in both training and test sets. We conclude with a 4-point protocol checklist for realistic IoT NIDS evaluation.

0.33

Towards Lightweight IoT Intrusion Detection: Feature Optimization and Explainable Machine Learning for Resource-Constrained Environment

cs.CR ·2026-11-07 ·Shaker Nawasra, Munther Abualkibash
published similarity 0.59
Abstract

Machine learning-based intrusion detection systems (IDSs) can improve IoT security but often rely on large feature sets that increase computational overhead and limit deployment on resource-constrained devices. This paper presents a resource-efficient and explainable feature optimization framework for lightweight IoT intrusion detection using the CICIoT2023 dataset. A Random Forest model was used to rank feature importance and identify compact feature subsets. Experimental results show that a three-feature model achieved an MCC of 0.9622 while reducing the feature space by 92.1%, whereas a six feature model achieved an MCC of 0.9675 with an 84.2% reduction. Furthermore, a fifteen-feature model achieved an MCC of 0.9744, preserving 99.97% of the performance of the full 38-feature baseline model while reducing the feature space by 60.5%. SHAP-based explainability analysis validated the selected features and provided interpretable insights into model decisions. The results demonstrate that lightweight, explainable, and high-performance IoT intrusion detection can be achieved using only a small subset of carefully selected features, significantly reducing resource requirements with minimal loss of detection effectiveness.

0.20