CS Paper Radar

Multi-Class, Multi-Tier Network Intrusion Detection: A Comprehensive and Reproducible Benchmark

Yufeng Xin, Bryant Goseland, Mohamed Rahouti

cs.CR cs.LG ·submitted 2026-09-28 18:06 UTC · v1
benchmarkcode/data

Abstract

Machine learning (ML) and deep learning (DL) have dominated Intrusion Detection System (IDS) research in recent years. Unfortunately, many existing studies have produced inflated results and unreliable benchmarks due to critical oversights and mistakes in the ML and DL pipeline, from data collection and labeling to feature engineering and model training and evaluation. CIC-IDS2017 is a standard benchmark for network intrusion detection. Still, many published results on this dataset are difficult to compare due to labeling errors, inconsistent flow extraction, potential leakage, and performance evaluation metrics dominated by benign traffic. In this paper, we present a comprehensive benchmark with corrected PCAP-level labeling and a complete evaluation pipeline with diverse ML models. We evaluate eleven tabular classifiers at three nested levels: binary attack detection, nine-class attack-family attribution, and fifteen-class fine-grained classification. A soft-voting ensemble of Random Forest, XGBoost, and LightGBM obtains the best fine-tier macro-F1 of 0.955, with coarse and binary macro-F1 scores of 0.980 and 0.999, respectively. We further conducted a feature selection study based on an analysis of feature importance. This comprehensive benchmark pipeline is configurable and open-source, enabling new feature extraction and model plugins for new datasets. Future work should use this pipeline as a reference point for richer features, rare-class analysis, and model generalization towards new datasets and attack classes.

Closest papers

Nearest neighbours by meaning across the whole index.

Evaluating ML-based Intrusion Detection Systems: The Illusion of Model Efficacy

cs.CR ·2026-09-02 ·Achilleas Spanos, Ioanna Kantzavelou
similarity 0.74
Abstract · 19 pages, 6 figures, 3 tables

Intrusion Detection has been revolutionized due to the integration of Machine Learning. Improved detection rates, reduced false alarms, and optimized algorithms contribute to the perception of improved systems with optimal accuracy and near-perfect performance, the illusion of model efficacy. However, the value of this effectiveness diminishes when confronted with unseen attacks. In this paper, we go beyond solely algorithmic enhancements and metric adjustments in ML-based Network Intrusion Detection Systems. We design an experiment to test the generalization capabilities of certain classifiers on unseen attacks. Our approach examines the dimensionality parameter's impact through two experimental methodologies, which are applied in two distinct settings. The experimental findings reveal how effectively the models could identify even a fraction of unseen attacks and underscore structural weaknesses in ML-based IDS research and evaluation techniques. Finally, seven evaluation criteria are outlined to address these challenges.

0.29

Few-Shot Learning for Network Intrusion Detection: Methods, Datasets, and Performance

cs.CR ·2026-09-10 ·Arne Roszeitis, Victor Jüttner, Erik Buchmann
survey similarity 0.74
Abstract

Anomaly-based network intrusion detection systems (NIDS) are an important first line of defense. However, training NIDS for new attack types is challenging, because labeled attack data are rarely available. Few-shot learning (FSL) addresses this problem by learning from few samples. However, the approaches and evaluation settings, that have been investigated so far, vary widely. This work systematically reviews FSL approaches for NIDS published from 2022 to 2026. We conduct a systematic literature review with PRISMA 2020-like reporting to search ACM Digital Library, IEEE Xplore, and Scopus. From a set of 1,358 initial records, we retain 21 studies after screening, deduplication, and quality filtering. We classify the applied FSL approaches, datasets, and experimental parameters and compare reported performance. Meta-learning and convolutional neural networks are the most common approaches, with 8 and 10 studies, respectively. Most studies evaluate five or fewer samples per class, although settings vary. CIC-IDS2017 and CSE-CIC-IDS2018 are the most frequently used datasets. Missing parameters and source code limit reproducibility and direct comparison between approaches.

0.25

Network Intrusion Detection Adaptability under Spurious Labels: A Noise-Aware Few-Shot Class-Incremental Learning Framework

cs.NI cs.AIcs.LG ·2026-11-16 ·Zijian Zhao, Davide Di Monda, Domenico Ciuonzo, Yipeng Wang, Antonio Pescapè
publishedcode/datahot direction similarity 0.74
Abstract

Network Intrusion Detection Systems (NIDSs) must incrementally adapt to emerging attacks, even when only a limited amount of labeled network traffic is available. Consequently, Few-Shot Class-Incremental Learning (FSCIL)-based NIDSs have gained growing attention. However, existing FSCIL-based NIDSs are highly vulnerable to label noise in newly-observed attack classes. To address this limitation, we propose NAFN, a noise-aware FSCIL-based NIDS framework that reduces the impact of mislabeled data during incremental adaptation. NAFN consists of two components: (i) Noise Detection, which identifies and removes mislabeled biflows using noise-detection algorithms, and (ii) Feature Sampling, which estimates class-conditional embeddings from clean biflows, calibrates them with base-class statistics, and resamples synthetic features to recover the original biflow count. The framework follows the Rethink Few-Shot (RFS) teacher-student paradigm with lightweight nearest-neighbor or logistic regression classifiers, and various pretraining & memory selection strategies. Experiments on CSE-CIC-IDS2018, Edge-IIoT, and IoT-NID datasets show that NAFN effectively preserves prior knowledge and maintains strong incremental performance even under 50% label noise, consistently improving new-class F1-scores over non-denoised baselines, up to +52%, while being time-efficient and robust in cross-network and high-noise incremental settings. The implementation is publicly available to foster reproducibility.

0.44

Dueling Deep Q-Learning for Intrusion Detection

cs.CR cs.LG ·2026-08-11 ·Logan Luna, Matthew P. Berkowitz, Laxima Niure Kandel, Sirio Jansen-S'anchez
publishedbenchmark similarity 0.74
Abstract · 6 pages, 5 figures. Published in Proc. IEEE SoutheastCon 2025, pp. 1192-1197

Intrusion detection systems (IDS) and automated systems for detecting and reporting cyber threats, are commonly handled via supervised machine learning methods. Though effective, these models struggle to effectively adapt to new attack types. This study proposes a novel approach by employing a reward-based, dueling Q-learning model for IDS, achieving an average accuracy of 99.68% across multiple attack classes. The proposed model has a dueling network architecture which separates its predictions into value and advantage streams. This has the benefit of improving learning efficiency and stability. The model was trained on the CIC-IDS2018, a benchmark dataset based on real-world intrusion detection scenarios, having multiple attack classes such as DDoS, botnets, and brute-force attacks. Furthermore, Explainable AI (XAI), specifically SHAP (SHapley Additive exPlanations), was also integrated into the training and evaluation process to provide interpretability into the model's predictions.

0.28

Adversarial Robustness Is Not Zero-Day Generalization: A Controlled Evaluation of Flow-Based Intrusion Detectors on Corrected CIC-IDS2017

cs.CR cs.LG ·2026-10-31 ·Abdulhamid Zaro
code/data similarity 0.74
Abstract · 10 pages, 10 tables. Code and per-seed results: https://github.com/abdulhamid-zaro/robustness-vs-zeroday-nids

Synthetic variants of known attacks, including feature-space adversarial perturbations, are sometimes used as stand-ins for unseen ("zero-day") attacks when evaluating machine-learning network intrusion detection systems (NIDS). We test whether this proxy holds. On the corrected CIC-IDS2017 release we train Random Forest, XGBoost, a 1D-CNN and an FT-Transformer and compare, at a fixed 1% false-positive rate over five seeds, detection under constrained perturbations of known attacks (gradient-based and black-box attacks on 42 attacker-controllable features) with leave-one-attack-family-out (LOAO) detection, where an entire family is absent from training. Detection under the strongest perturbation we tested, with budgets of at most 0.2, stays between 90.9% and 99.7%, yet mean detection of held-out families falls to 76.2% (RF), 40.2% (XGBoost), 25.8% (1D-CNN) and 22.8% (FT-Transformer). Robustness exceeds LOAO detection for every model (gaps of 24 to 72 percentage points, 23 to 66 when robustness is also averaged over families; Holm-adjusted p $\le$ 0.00096), and the two measures agree at most on a coarse ranking of the models. An exact mixed-integer evasion search confirms the tree ensembles' scores. Benign-only anomaly detectors and a simple multi-class open-set baseline do not close the gap. A temporal split on 5 seeds (3 of the 4 models) leaves the gap in place. On CSE-CIC-IDS2018 the gap recurs for every model (16 to 68 points). On these benchmarks and under this threat model, a robustness score should not be read as a zero-day score, and the two should be reported separately. Code and per-seed results are publicly available.

0.40

A Calibrated and Explainable Bimodal Machine Learning Framework for Hybrid Intrusion Detection

cs.CR ·2026-08-17 ·Hafsa Aslam, Yue Li, Saba Aslam, Gracious Mwamughunda
top venue similarity 0.73
Abstract

Modern communication systems face critical gaps in detecting unknown attacks and rare threat classes due to extreme data imbalance and black-box decision logic. We propose a bimodal framework of calibrated and explainable machine learning (ML) for network security, unifying known-class precision with open-set generalization without the complexity of deep learning. Our framework introduces security-oriented feature extraction to enhance signal-to-noise ratio, hybrid resampling (ADASYN + manual boosting) to reduce class imbalance, isotonic calibration and adaptive thresholding ($τ=0.30$ for XSS) to recover recall for rare attacks, and SHAP-based explainability to validate domain-aligned decision logic. Evaluated on the CIC-IDS2017 dataset and compared with prior ML models and studies, our framework achieves significant accuracy on known attacks (Macro F1 = 0.8626) and detects unknown classes at 1% FPR with TPR up to 90.17% (DoS slowloris), and 77.04% (Web-XSS). The SHAP analysis confirms decisions are driven by security-relevant features, not model artifacts. Our work bridges the gap between theoretical models and operational IDS by delivering calibrated, explainable, and open-set-capable attack detection and prevention in a single, reproducible framework. Keywords: intrusion detection, cybersecurity and privacy, explainable AI, machine learning

0.24

DeepWeave: Weaving Neural and Symbolic Evidence for ML-Based Network Intrusion Detection System

cs.CR cs.LGcs.NI ·2026-10-30 ·Raoul Saad, Enkeleda Bardhi, Andrea Agiollo, Fernando Kuipers
similarity 0.73
Abstract

Machine learning-based network intrusion detection systems can recognize complex traffic patterns, but often require substantial labelled attack data, degrade under deployment shift, and provide limited evidence for analysts. We present DeepWeave, a probabilistic neuro-symbolic Network Intrusion Detection System (NIDS) that makes logic an integral part of detection. DeepWeave automatically constructs and validates interpretable rules, represents neural predictions as probabilistic predicates, and reconciles both evidence sources through explicit coupling policies over packet-, flow-, and hybrid-views. The resulting proofs provide intrinsic explanations for rule-supported alerts. Across multiple datasets, DeepWeave improves low-data detection, false-positive control, transferability, and robustness. With only 50 labelled packet observations per attack class, it improves macro-F1 from 0.632 to 0.924 and reduces the attack false-positive rate from 0.075 to 0.008 over the matched neural baseline. Its proofs are also more stable than post-hoc neural attributions, while batched inference keeps probabilistic reasoning practical. These results demonstrate label-efficient, adaptable, and inspectable neuro-symbolic intrusion detection.

0.39

Adaptive and Proximal Federated Optimization for Transferable Network Intrusion Detection

cs.CR ·2026-10-14 ·Rezwan Islam Salvi, MD Rafin Jawad Hafiz, Abeer Mahmood Kalam, Md Moniruzzaman
hot direction similarity 0.72
Abstract · 5 pages, 2 figures

Network Intrusion Detection Systems (NIDS) based on deep learning often fail to detect attack classes that are absent from training data, limiting their robustness against zero-day threats. This property called transferability across attack classes, remains underexplored especially in federated learning (FL) settings where data heterogeneity and client drift pose additional challenges. Prior work (TabFIDS) showed that FedAvg combined with bootstrapping and temporal averaging improves transferability for cross-device NIDS. In this paper, we explore whether other federated optimizers can do better. We compare two optimizers inspired by FedAdam and FedProx against the FedAvg baseline on the CIC-IDS 2017 dataset using an 11-client federated setup, sweep 10 hyperparameter configurations, and contrast two data-balancing strategies: bootstrapping and SMOTE. Using the count of client pairs whose transferred-model accuracy exceeds 70% as our primary metric (following TabFIDS), the proximal optimizer with a strong regularization term yields 48 transferable pairs versus 42 for FedAvg, while the adaptive optimizer with an aggressive server learning rate yields 46 pairs. SMOTE improves per-pair accuracy over bootstrapping but reduces breadth, revealing a depth-vs-breadth trade-off: high accuracy on a few client pairs versus moderate accuracy across most. These results indicate that the choice of federated optimizer, the rule by which client updates are regularized and aggregated into the global model, is very important for federated NIDS.

0.25