CS Paper Radar

AutoDP-LLM: Automating Data Pre-processing for Intrusion Detection Systems using Large Language Models

Bao-Phong Nguyen, Gia-Khanh Pham, Thai-Duong Do, Mai Xuan Trang, Minh-Tuan Le, Xuan-Nam Tran, Huan Vu, Tien-Cuong Nguyen, Vu-Duc Ngo, Thien Van Luong

cs.CR cs.AI ·submitted 2026-10-04 16:50 UTC · v1
top venuebenchmark

Abstract

The increasing complexity and scale of modern cyber-attacks demand intelligent and computationally efficient Intrusion Detection Systems (IDS). However, designing effective data pre-processing pipelines traditionally involves substantial trial-and-error effort and repeated evaluation of alternative configurations. For large, high-dimensional network traffic data, this process can create a significant computational burden. In this work, we propose AutoDP-LLM, an automated pre-processing framework designed to reduce manual pipeline development and computational overhead. Specifically, AutoDP-LLM leverages Large Language Models (LLMs) to autonomously generate and validate executable data pre-processing pipelines. The framework combines deterministic host-side planning with LLM-based specialist agents to formulate data-processing strategies, synthesize executable code, and adaptively determine retained feature sets using semantic reasoning and training-derived statistical evidence, without requiring a predefined feature budget. Focusing on multiclass intrusion detection, we evaluate AutoDP-LLM on the UNSW-NB15 and NSL-KDD benchmark datasets using multiple downstream classifiers. Comparative experiments against conventional feature-selection methods show that AutoDP-LLM achieves competitive detection performance while automating the generation of compact and executable pre-processing pipelines. Component-level ablation experiments further demonstrate the complementary contributions of the semantic and statistical feature-reduction components. The repeated generation, validation, execution, and assessment of candidate pipelines are amenable to parallel execution, highlighting the potential of scalable computing environments, including high-performance computing (HPC) systems, to support automated IDS pipeline development.

Journal ref
The Journal of Supercomputing 2026
DOI
10.1007/s11227-026-08894-8

Closest papers

Nearest neighbours by meaning across the whole index.

Multi-Class, Multi-Tier Network Intrusion Detection: A Comprehensive and Reproducible Benchmark

cs.CR cs.LG ·2026-09-28 ·Yufeng Xin, Bryant Goseland, Mohamed Rahouti
benchmarkcode/data similarity 0.70
Abstract

Machine learning (ML) and deep learning (DL) have dominated Intrusion Detection System (IDS) research in recent years. Unfortunately, many existing studies have produced inflated results and unreliable benchmarks due to critical oversights and mistakes in the ML and DL pipeline, from data collection and labeling to feature engineering and model training and evaluation. CIC-IDS2017 is a standard benchmark for network intrusion detection. Still, many published results on this dataset are difficult to compare due to labeling errors, inconsistent flow extraction, potential leakage, and performance evaluation metrics dominated by benign traffic. In this paper, we present a comprehensive benchmark with corrected PCAP-level labeling and a complete evaluation pipeline with diverse ML models. We evaluate eleven tabular classifiers at three nested levels: binary attack detection, nine-class attack-family attribution, and fifteen-class fine-grained classification. A soft-voting ensemble of Random Forest, XGBoost, and LightGBM obtains the best fine-tier macro-F1 of 0.955, with coarse and binary macro-F1 scores of 0.980 and 0.999, respectively. We further conducted a feature selection study based on an analysis of feature importance. This comprehensive benchmark pipeline is configurable and open-source, enabling new feature extraction and model plugins for new datasets. Future work should use this pipeline as a reference point for richer features, rare-class analysis, and model generalization towards new datasets and attack classes.

0.18

An LLM-Assisted AutoML Framework for Intrusion Detection in IoT Networks

cs.CR cs.LG ·2026-09-19 ·Li Yang
code/data similarity 0.69
Abstract · Submitted to an IEEE Journal. Code will be released at: https://github.com/LiYangHart/LLM-Assisted-AutoML-For-Intrusion-Detection

Internet of Things (IoT) systems are increasingly deployed in smart homes, transportation, energy systems, and critical infrastructure. This broad connectivity improves service intelligence, but also enlarges the attack surface of IoT networks. Machine Learning (ML)-based Intrusion Detection Systems (IDSs) are widely used to identify malicious network threats and protect IoT systems, but developing effective ML-based IDS models often requires human expertise and repeated manual decisions on many procedures, including data pre-processing, feature selection, model selection, and hyperparameter tuning. Automated Machine Learning (AutoML) reduces this burden by automating steps of the ML pipeline using optimization techniques, but conventional AutoML methods can consume substantial optimization time because they explore broad candidate model families and large hyperparameter spaces. This paper proposes a Large Language Model (LLM)-assisted AutoML framework for IoT intrusion detection. The proposed framework uses an LLM as a policy generator that converts dataset profiles into bounded and validated AutoML policies for automated data balancing, automated feature engineering, and Combined Algorithm Selection and Hyperparameter Optimization (CASH). Under an equal 10-trial budget, the proposed LLM-assisted policy achieves higher weighted test F1-score than traditional AutoML using the Tree-structured Parzen Estimator (TPE) on both datasets, reaching 99.680% on CICIDS2017 and 99.186% on IoTID20. Relative to the broader 30-trial Traditional AutoML-TPE baseline, the 10-trial proposed method reduces optimizer time by 63.7% and 49.9%, respectively, while achieving slightly higher F1-score. These results show that a bounded LLM policy can improve the quality of a low-budget AutoML search while retaining a clear efficiency advantage relative to a larger conventional search budget.

0.40

DeepWeave: Weaving Neural and Symbolic Evidence for ML-Based Network Intrusion Detection System

cs.CR cs.LGcs.NI ·2026-10-30 ·Raoul Saad, Enkeleda Bardhi, Andrea Agiollo, Fernando Kuipers
similarity 0.68
Abstract

Machine learning-based network intrusion detection systems can recognize complex traffic patterns, but often require substantial labelled attack data, degrade under deployment shift, and provide limited evidence for analysts. We present DeepWeave, a probabilistic neuro-symbolic Network Intrusion Detection System (NIDS) that makes logic an integral part of detection. DeepWeave automatically constructs and validates interpretable rules, represents neural predictions as probabilistic predicates, and reconciles both evidence sources through explicit coupling policies over packet-, flow-, and hybrid-views. The resulting proofs provide intrinsic explanations for rule-supported alerts. Across multiple datasets, DeepWeave improves low-data detection, false-positive control, transferability, and robustness. With only 50 labelled packet observations per attack class, it improves macro-F1 from 0.632 to 0.924 and reduces the attack false-positive rate from 0.075 to 0.008 over the matched neural baseline. Its proofs are also more stable than post-hoc neural attributions, while batched inference keeps probabilistic reasoning practical. These results demonstrate label-efficient, adaptable, and inspectable neuro-symbolic intrusion detection.

0.39

Network Intrusion Detection Adaptability under Spurious Labels: A Noise-Aware Few-Shot Class-Incremental Learning Framework

cs.NI cs.AIcs.LG ·2026-11-16 ·Zijian Zhao, Davide Di Monda, Domenico Ciuonzo, Yipeng Wang, Antonio Pescapè
publishedcode/data similarity 0.67
Abstract

Network Intrusion Detection Systems (NIDSs) must incrementally adapt to emerging attacks, even when only a limited amount of labeled network traffic is available. Consequently, Few-Shot Class-Incremental Learning (FSCIL)-based NIDSs have gained growing attention. However, existing FSCIL-based NIDSs are highly vulnerable to label noise in newly-observed attack classes. To address this limitation, we propose NAFN, a noise-aware FSCIL-based NIDS framework that reduces the impact of mislabeled data during incremental adaptation. NAFN consists of two components: (i) Noise Detection, which identifies and removes mislabeled biflows using noise-detection algorithms, and (ii) Feature Sampling, which estimates class-conditional embeddings from clean biflows, calibrates them with base-class statistics, and resamples synthetic features to recover the original biflow count. The framework follows the Rethink Few-Shot (RFS) teacher-student paradigm with lightweight nearest-neighbor or logistic regression classifiers, and various pretraining & memory selection strategies. Experiments on CSE-CIC-IDS2018, Edge-IIoT, and IoT-NID datasets show that NAFN effectively preserves prior knowledge and maintains strong incremental performance even under 50% label noise, consistently improving new-class F1-scores over non-denoised baselines, up to +52%, while being time-efficient and robust in cross-network and high-noise incremental settings. The implementation is publicly available to foster reproducibility.

0.50

IDSForge: Self-Explaining Network Intrusion Detection Through Neural Behavioral Rule Learning

cs.CR ·2026-10-28 ·Hao Fu, Jinxia Wei, Wan Wei, Chun Long, Degang Sun +1 more
similarity 0.66
Abstract

Anomaly-based network intrusion detection systems (NIDS) can detect previously unseen attacks by identifying deviations from benign traffic. However, deep learning-based anomaly detectors typically rely on implicit decision boundaries, and post-hoc explanations may not faithfully reflect the detection logic. This makes alerts difficult to interpret and audit. To address this issue, we present IDSForge, an anomaly-based NIDS that directly learns explicit, executable behavioral rules from benign traffic, making its detection logic interpretable by design. IDSForge uses the same rule set for detection and explanation, ensuring explanation fidelity by construction. It uses an explicit distributional autoencoder trained only on benign samples to learn axis-aligned regions of benign traffic. A high-speed rule compiler then consolidates these instance-specific regions into a compact set of feature-level interval constraints. These rules can be executed directly by popular NIDS engines, supporting high-throughput inspection with low overhead. To adapt to changes in traffic, a self-distillation co-evolver updates the model and rule set incrementally when the false-positive rate on analyst-verified traffic exceeds a target threshold. Our evaluation shows that IDSForge reduces F1 error by $5.7\times$ to $47\times$ relative to state-of-the-art anomaly-based baselines. Deployed in Suricata, IDSForge sustains a throughput of 7.8 Gbps with a median per-packet latency overhead of 3.1 $μ\text{s}$.

0.23

FSNIC: A Low-Latency Flow-Based Intrusion Detection Architecture for FPGA SmartNICs

cs.AR ·2026-09-14 ·Nise O'Cuill, Changhong Li, Georgios Floros, Shreejith Shanker
published similarity 0.66
Abstract · Accepted by ICECS'26

Modern data centres require high-performance networking alongside effective real-time security. Traditional Intrusion Detection Systems (IDS) commonly rely on general-purpose processors and often struggle to inspect high-speed traffic at line rate without introducing latency or performance bottlenecks. Smart Network Interface Cards (NICs) provide an alternative by enabling computation directly within the network data plane. This work presents a machine learning-based IDS implemented within an FPGA-based SmartNIC pipeline. The system integrates P4-based packet parsing with a LogicNets IDS model implemented in RTL, enabling deterministic, low-latency inference. Compared with traditional stateless packet-level classifiers, the proposed stateful flow-based IDS introduces minimal state by aggregating features across packets, capturing behavioural patterns not observable at the packet level. Experimental results on the UNSW-NB15 dataset show that the flow-based IDS improves detection accuracy from 86.92\% to 97.68\% compared with stateless packet-level classification. We also evaluate the proposed IDS on CICIDS2017 and compare its real-time hardware performance with prior FPGA-based IDS designs. Through hardware-software co-design, the proposed IDS achieves 6~ns inference latency using only 846 LUTs, with no BRAM or DSP usage, demonstrating a low latency and resource efficient implementation.

0.44

SERA-IDS: Structured Experience Retrieval-Augmented Intrusion Detection with Small Language Models

cs.CR ·2026-10-02 ·Md Shamimul Islam, Ayesha S. Dina
similarity 0.65
Abstract

Large language models (LLMs) offer a flexible approach to network intrusion detection, but direct classification of numerical flow records can be unreliable without traffic- specific decision boundaries. Retrieval-augmented generation (RAG) enables LLMs to leverage external experience; however, free-form textual experience requires the model to infer the numerical conditions that distinguish traffic classes. We propose SERA-IDS, a Structured Experience Retrieval-Augmented Intru- sion Detection framework that learns decision knowledge from classification errors. A Small Language Model (SLM) analyzes misclassified flows and generates structured rules containing behavioral descriptions, feature conditions, class-confusion infor- mation, tool-derived confidence, and provenance. A confidence gate admits supported rules into the Experience Library. During testing, the library is frozen, and retrieved class-diverse rules are condition-matched to the query flow and provided as evidence to the decision SLM. The same library is evaluated with three small, locally deployable SLMs, namely Llama 3.1, Phi-4:14B, and Qwen2.5:7B, without fine-tuning or paid hosted inference. On the test sets, SERA-IDS improves macro F1 on NF-BoT- IoT from 11.66%, 9.26%, and 7.83% to 86.46%, 69.50%, and 83.51%, respectively, and on NF-ToN-IoT from 3.82%, 6.17%, and 1.94% to 84.16%, 87.66%, and 83.37%. These results show that structured experience retrieval enables small, free, and locally deployable SLMs to achieve strong intrusion-detection performance, including performance exceeding that of the re- ported existing work used in our comparison.

0.29

Autonomous Cyber Defense: Real-Time Attack Detection and Mitigation in Software-Defined Networks Using Machine Learning

cs.CR cs.LG ·2026-08-22 ·Alexandre Amaral, Fernando Moro, Ana Malheiro
similarity 0.65
Abstract

Autonomous response has evolved into a timing-critical challenge rather than solely a matter of detection accuracy. In recent intrusions, the interval between initial access and the first lateral movement has been observed to be as short as 27 seconds, a window that precludes any human-in-the-loop workflow. This paper presents a closed-loop framework that detects and blocks attacks in software-defined networks without operator involvement, evaluating its performance against this stringent temporal constraint rather than relying exclusively on detection accuracy. An automated data pipeline collects IP flows and aggregates them into labeled training data, while a prevention module selects and trains candidate classifiers and issues blocking rules directly to the SDN controller. In a SYN flooding denial of service case study, the deployed K-Nearest Neighbors classifier achieved an F1 score of 96.7% and the cycle from flow availability to enforced block completed in 21 seconds, below the fastest breakout time reported to date.

0.26