CS Paper Radar

GreenNIDS: A Benchmark Workflow and Analysis Dataset for NetFlow Intrusion Detection Experiments

Lilia-Eliana Popescu-Apreutesei, Sabina-Cristiana Necula

cs.CR cs.LG ·submitted 2026-10-18 16:11 UTC · v1
publishedbenchmark

Abstract

Comparing the predictive performance and computational cost of flow-based intrusion detection experiments requires both quantities to be available for each run. GreenNIDS connects benchmark execution on public NetFlow datasets to a reusable analysis dataset. Model implementations write a common set of outputs, which are ingested into a run-level table with companion tables for model configuration, feature importance, and training history. The run-level table combines predictive metrics with timing, throughput, memory use, and energy estimates based on thermal design power. A CPU-only demonstration comprises 216 runs across four dataset families, three versions, and binary and multiclass tasks. Ingestion produces a 216 x 96 analysis table with no duplicate run identifiers and complete Matthews correlation coefficient (MCC) and F1 coverage. Multiclass ROC-AUC is missing for 36 PyTorch runs, making metric availability an explicit part of subsequent comparisons. The stored records support grouped performance and training-time plots, as well as resource screening after execution. Combining task-specific MCC thresholds with limits on training time, peak RAM, and estimated training energy retains 25.0% of the full run collection. These results establish compatibility within the evaluated collection and support comparisons after execution, subject to the recorded preprocessing, execution profile, and approximate energy accounting.

Comment
12 pages, 3 figures, 2 tables. Accepted at IE 2026 (International Conference on Informatics in Economy)

Closest papers

Nearest neighbours by meaning across the whole index.

Unmasking Shortcut Learning in IoT Intrusion Detection: A Forensic, Multi-Paradigm Evaluation of Feature Dependence and Data Leakage

cs.CR cs.LGcs.NI ·2026-09-23 ·Uday Shankar Roy, Mahbuba Jahan Minu
benchmark similarity 0.65
Abstract · Currently under review

Machine learning-based Network Intrusion Detection Systems often report near-perfect performance on IoT benchmarks. However, whether these models learn generalizable attack behavior or exploit spurious dataset shortcuts- such as static testbed IP/MAC addresses and chronological recording artifacts-remains an important question. We evaluate the CyberFlowIoT-GICAP benchmark, containing 3,617,388 flow records across 126 PCAP sessions with 849,395 benign flows. Four learning paradigms are evaluated across four feature configurations using PCAP-disjoint splits; LightGBM is additionally evaluated using conventional random-flow splitting. When only statistical flow behavior is used (Fbehav), LightGBM (92.58% +/- 8.18%), Random Forest (92.59% +/- 8.18%), and Deep MLP (92.55% +/- 8.18%) achieve nearly identical Macro-F1, indicating that performance is constrained by feature representation rather than model complexity. With raw timestamps (Ftstamp), tree-based models reach 99.28% Macro-F1, while the linear model remains at 90.62%, showing that nonlinear models can exploit dataset-specific temporal structure. Attack detectability is highly asymmetric: high-rate and active attacks maintain >99.8% recall from flow behavior alone in nonlinear models, whereas the DNS Beaconing drops from 27.78% to 0.00% recall when contextual features are removed. Conventional random-flow splitting increases attack recall by up to 14.00%, highlighting the effect of placing flows from the same sessions in both training and test sets. We conclude with a 4-point protocol checklist for realistic IoT NIDS evaluation.

0.13

Jev-IDS: System One Models for Network Intrusion Detection

cs.CR cs.AI ·2026-10-01 ·Paulo Severo, Silvio E. Quincozes, Amanda Dias
similarity 0.62
Abstract

Machine-learning Network Intrusion Detection Systems (IDS) depend on substantial labeled datasets and task-specific training, whereas Large Language Models (LLMs) detection can analyze flow records directly but incurs higher inference cost and latency, with less constrained outputs. This paper presents JEV-IDS, an open experimental general NIDS based on the Jev System One Model (SOM) to detect zero day intrusions Under label scarcity. JEV-IDS serializes one flow per request and asks JEV two questions: a binary attack probability and a finite-choice traffic category. Our results show that, at k=1, JEV was 4.8 times faster and 3.8 times cheaper than GPT-5.6 Luna, with 1.5 times higher novel-attack recall; it also produced 15 times fewer false alarms than a low-data Random Forest. Across 5,400 decisions on a 300-flow NSL-KDD pilot split, JEV achieved F1-Score 0.859, precision 0.941, recall 0.790, and novel-attack recall 0.838. Increasing k to 2 reduced its F1-Score to 0.839.

0.22

AutoDP-LLM: Automating Data Pre-processing for Intrusion Detection Systems using Large Language Models

cs.CR cs.AI ·2026-10-04 ·Bao-Phong Nguyen, Gia-Khanh Pham, Thai-Duong Do, Mai Xuan Trang, Minh-Tuan Le +5 more
top venuebenchmark similarity 0.62
Abstract

The increasing complexity and scale of modern cyber-attacks demand intelligent and computationally efficient Intrusion Detection Systems (IDS). However, designing effective data pre-processing pipelines traditionally involves substantial trial-and-error effort and repeated evaluation of alternative configurations. For large, high-dimensional network traffic data, this process can create a significant computational burden. In this work, we propose AutoDP-LLM, an automated pre-processing framework designed to reduce manual pipeline development and computational overhead. Specifically, AutoDP-LLM leverages Large Language Models (LLMs) to autonomously generate and validate executable data pre-processing pipelines. The framework combines deterministic host-side planning with LLM-based specialist agents to formulate data-processing strategies, synthesize executable code, and adaptively determine retained feature sets using semantic reasoning and training-derived statistical evidence, without requiring a predefined feature budget. Focusing on multiclass intrusion detection, we evaluate AutoDP-LLM on the UNSW-NB15 and NSL-KDD benchmark datasets using multiple downstream classifiers. Comparative experiments against conventional feature-selection methods show that AutoDP-LLM achieves competitive detection performance while automating the generation of compact and executable pre-processing pipelines. Component-level ablation experiments further demonstrate the complementary contributions of the semantic and statistical feature-reduction components. The repeated generation, validation, execution, and assessment of candidate pipelines are amenable to parallel execution, highlighting the potential of scalable computing environments, including high-performance computing (HPC) systems, to support automated IDS pipeline development.

0.39

Multi-Class, Multi-Tier Network Intrusion Detection: A Comprehensive and Reproducible Benchmark

cs.CR cs.LG ·2026-09-28 ·Yufeng Xin, Bryant Goseland, Mohamed Rahouti
benchmarkcode/data similarity 0.61
Abstract

Machine learning (ML) and deep learning (DL) have dominated Intrusion Detection System (IDS) research in recent years. Unfortunately, many existing studies have produced inflated results and unreliable benchmarks due to critical oversights and mistakes in the ML and DL pipeline, from data collection and labeling to feature engineering and model training and evaluation. CIC-IDS2017 is a standard benchmark for network intrusion detection. Still, many published results on this dataset are difficult to compare due to labeling errors, inconsistent flow extraction, potential leakage, and performance evaluation metrics dominated by benign traffic. In this paper, we present a comprehensive benchmark with corrected PCAP-level labeling and a complete evaluation pipeline with diverse ML models. We evaluate eleven tabular classifiers at three nested levels: binary attack detection, nine-class attack-family attribution, and fifteen-class fine-grained classification. A soft-voting ensemble of Random Forest, XGBoost, and LightGBM obtains the best fine-tier macro-F1 of 0.955, with coarse and binary macro-F1 scores of 0.980 and 0.999, respectively. We further conducted a feature selection study based on an analysis of feature importance. This comprehensive benchmark pipeline is configurable and open-source, enabling new feature extraction and model plugins for new datasets. Future work should use this pipeline as a reference point for richer features, rare-class analysis, and model generalization towards new datasets and attack classes.

0.18

Provenance, Not Behaviour: A Serialisation Artifact in Edge-IIoTset and a Leakage-Free Benchmark for Precision-Agriculture Intrusion Detection

cs.CR cs.LG ·2026-08-16 ·Mostafa M. Galal
benchmark similarity 0.60
Abstract · 29 pages, 13 tables. Code: https://doi.org/10.5281/zenodo.21941210 Benchmark: https://doi.org/10.5281/zenodo.21941319

Edge-IIoTset is the reference benchmark for machine-learning intrusion detection in the industrial Internet of Things, and results reported on it cluster above 99%. We show that much of that performance is not intrusion detection. The preprocessing recipe distributed with the dataset instructs researchers to one-hot encode seven categorical columns. Four of them separate attack from normal traffic with an accuracy of 1.0000 on their own, through the spelling of the placeholder written for an absent protocol field: the string "0" in the normal-traffic branch of the dataset build against "0.0" in the attack branch. The label is recoverable from a serialisation artifact encoding file provenance, with no network behaviour modelled, and separates every row of both curated subsets. Under 5-fold x 3-repeat cross-validation, five of six standard classifiers attain exactly 1.0000 +/- 0.0000 accuracy and the sixth attains 0.99998. Under a corrected protocol, naive Bayes falls by 0.3005 macro-F1 and the strongest model settles at 0.9503 +/- 0.0011. Label, ordinal and frequency encoding leak identically. Because the curated subsets also lack Modbus and per-device identity, we rebuild the benchmark from the raw captures under uniform parsing, producing AgriEdge: 1,276,122 rows, five devices with full attribution, and no column separating the classes above 0.0288. A leave-one-device-out sweep locates the generalisation boundary at the perception/actuation layer, where random forest falls from 0.9988 to 0.5083 balanced accuracy. Non-IID federated partitioning costs at most 0.0037 macro-F1, but a 20-round LoRaWAN training run costs 4.6 hours of uplink.

0.22

Hierarchical Personalized Federated Learning for Fine-Grained Intrusion Detection: A Multi-Class Analysis in Heterogeneous Attack Environments

cs.CR ·2026-11-16 ·Abul Kalam Azad, Travis Atkison
published similarity 0.60
Abstract · Accepted to present at the 2026 IEEE International Conference on Cybersecurity Systems, Technologies and Applications (CSTA)

Intrusion Detection Systems (IDS) play a vital role in protecting modern network infrastructures. Although Federated Learning (FL) enables collaborative training without sharing raw network traffic, global models struggle with non-IID attack distributions. Moreover, practical security operations require fine-grained attack classification for targeted threat responses. To address these challenges, this paper investigates the convergence dynamics and performance of Hierarchical Personalized Federated Learning (HPFL), which combines multi-tier aggregation with client-level personalization, for 8-class, 6-class, and 4-class intrusion detection using Multi-Layer Perceptron (MLP) models implemented in NVIDIA FLARE on the CICIDS2017 dataset. Experimental results show that personalized HPFL models achieve micro F1-scores up to 0.9911 in the 6-class task, while the overall local accuracy reaches 0.9975 in the 8-class task. On 8-class local test sets, HPFL achieves a micro F1-score of 0.9901 and a macro F1-score of 0.8324, outperforming FedAvg (0.7176) and FedProx (0.5624) in macro F1. Cross-client evaluations demonstrate out-of-distribution generalization, with HPFL achieving a 4-class micro F1-score of 0.9294 compared with 0.9140 for standalone local models. Furthermore, distance-weighted network analysis estimates a 48.9% reduction in transmission overhead compared with conventional FL (194.4 MB vs. 380.3 MB for the 8-class task). These findings demonstrate HPFLs potential as an accurate, privacy-preserving, and communication-efficient approach to fine-grained intrusion detection in heterogeneous networks.

0.41

Replay-Augmented Regularisation for Class-Incremental Network Intrusion Detection: A Three-Dataset Evaluation of EWC and LwF

cs.CR ·2026-11-16 ·Mohammed Fackruddin Tusher, Muhammad Iqbal Hossain
similarity 0.59
Abstract

As new attack categories emerge, network intrusion detection models need to be updated; however, when fine-tuned sequentially on new classes, previous classes are forgotten, resulting in catastrophic for- getting. The three public datasets we use are UNSW-NB15, ToN-IoT and CIC-IDS2017, and for these datasets, our class-incremental intrusion detection system is built using a CNN with one-dimensional (1D) archi- tecture trained over nine sequential tasks, each of which introduces one attack category. We evaluate sequential fine-tuning without any buffe compared to Elastic Weight Consolidation (EWC) and Learning without Forgetting (LwF), both of which include a class-size-adaptive rehearsal buffer of 20% of all previous classes (from 20 to 300 samples). The hyper-parameters were selected using UNSW-NB15 and fixed for the other two datasets. On UNSW-NB15, replay improves the final accuracy of both regularisers to 73.6% (EWC) and 71.1% (LwF) compared to about 32% (no improvement over fine-tuning). LwF with replay achieves 46.1% and 91.6% final accuracy for the two frozen-configuration datasets (ToN-IoT, CIC-IDS2017) while EWC with replay achieves 23.7% and 61.9%. In order to avoid this, EWC was therefore tuned on one dataset and the other two data sets were used as test sets. It is also demonstrated that the common forgetting measure can be misleading for a model which does not learn the old classes. As the fine-tuning baseline was not re- played and all the results were based on a single seed, it is not claimed hat the regularisers provide any extra support over replay alone.

0.33

Adaptive and Proximal Federated Optimization for Transferable Network Intrusion Detection

cs.CR ·2026-10-14 ·Rezwan Islam Salvi, MD Rafin Jawad Hafiz, Abeer Mahmood Kalam, Md Moniruzzaman
hot direction similarity 0.59
Abstract · 5 pages, 2 figures

Network Intrusion Detection Systems (NIDS) based on deep learning often fail to detect attack classes that are absent from training data, limiting their robustness against zero-day threats. This property called transferability across attack classes, remains underexplored especially in federated learning (FL) settings where data heterogeneity and client drift pose additional challenges. Prior work (TabFIDS) showed that FedAvg combined with bootstrapping and temporal averaging improves transferability for cross-device NIDS. In this paper, we explore whether other federated optimizers can do better. We compare two optimizers inspired by FedAdam and FedProx against the FedAvg baseline on the CIC-IDS 2017 dataset using an 11-client federated setup, sweep 10 hyperparameter configurations, and contrast two data-balancing strategies: bootstrapping and SMOTE. Using the count of client pairs whose transferred-model accuracy exceeds 70% as our primary metric (following TabFIDS), the proximal optimizer with a strong regularization term yields 48 transferable pairs versus 42 for FedAvg, while the adaptive optimizer with an aggressive server learning rate yields 46 pairs. SMOTE improves per-pair accuracy over bootstrapping but reduces breadth, revealing a depth-vs-breadth trade-off: high accuracy on a few client pairs versus moderate accuracy across most. These results indicate that the choice of federated optimizer, the rule by which client updates are regularized and aggregated into the global model, is very important for federated NIDS.

0.25