CS Paper Radar

Beyond Model Scaling: Rethinking Sequence Construction for Transformer-based Network Intrusion Detection

Zeeshan Zulkifl Shah, Muhammad Ikram, Dali Kaafar, Yipeng Zhou

cs.CR cs.NI ·submitted 2026-10-13 04:58 UTC · v2
code/data

Abstract

Transformer-based network intrusion detection systems (NIDS) have shown strong potential for detecting multi-stage cyberattacks, yet most methods construct input sequences using fixed temporalization schemes rather than treating sequence construction as a systems design decision. In flow-based intrusion detection, timestamp-based windowing often fragments related attack activities while merging unrelated communications, reducing the behavioral context available to transformers. We present a unified pipeline abstraction that decomposes transformer-based NIDS into four design dimensions (traffic representation, temporalization, transformer backbone, and prediction aggregation) and propose session-coherent temporalization, which constructs sequences from communication sessions instead of fixed time windows. Experiments on three complementary intrusion detection datasets show that temporalization is the dominant factor governing transformer performance: session-coherent temporalization more than doubles Macro-F1 over timestamp-based windowing, enabling lightweight transformers to achieve up to 0.95 Macro-F1 at substantially higher inference throughput than pretrained models. We further vary the transformer size from 0.65M to 3.38M parameters and find that larger transformer models provide no measurable accuracy improvement despite substantially higher computational cost, while zero-shot cross-dataset evaluation shows only modest gains under domain shift. These results indicate that preserving communication-session semantics, rather than scaling transformer capacity, is the key to accurate and efficient transformer-based intrusion detection.

Comment
11 pages, 8 figures, 5 tables. Reproducibility artifact: https://github.com/ZeeshanZulkiflShah/sequence-construction-nids-artifact/tree/arxiv-v1-artifact

Closest papers

Nearest neighbours by meaning across the whole index.

Federated Lightweight Transformer for Network-Layer Cyberattack Detection in O-RAN

cs.CR cs.AI ·2026-11-03 ·Hong Nhung Nguyen, Abdulahi Badrudeen, Yekaterina Kim, Truong Thu Huong, Sunwoo Kim
published similarity 0.69
Abstract · Accepted to RIVF conference

Open Radio Access Network (O-RAN) improves radio access network flexibility through disaggregation, open interfaces, and programmable RAN Intelligent Controllers (RICs). However, the same openness and distribution increase the attack surface and make centralized intrusion detection difficult to scale across heterogeneous RAN sites. This paper presents a federated lightweight Transformer framework for privacy-preserving network-layer cyberattack detection in O-RAN. Each participating site trains a compact Transformer locally on temporally ordered network-flow records, while only model parameters are shared with a federated coordinator. To model realistic site heterogeneity, the client datasets are partitioned according to a Dirichlet distribution, resulting in non-independent and identically distributed (non-IID) local data. The local detector uses feature projection, positional encoding, multi-head self-attention, feed-forward layers, and temporal pooling to classify five network-layer attack categories: ICMP, Slowloris, SYN, TCP-ACK, and UDP. Experiments on the NetsLab 5G O-RAN intrusion-detection dataset show an overall accuracy of 94.89 % and a macro-F1 score of 81.09 %.

0.27

Event-Native Symbolic-Temporal Spike Encoding Framework for Heterogeneous Cyber Streams

cs.NE cs.AIcs.CR ·2026-09-14 ·Dalton Diez, Peyton Andras, Max Shroyer, James Ghawaly
published similarity 0.66
Abstract · Accepted in The 38th IEEE International Conference on Tools with Artificial Intelligence (ICTAI)

Spiking neural networks (SNNs) have shown promise for sparse, event-driven computation through stateful processing that is naturally compatible with low-power edge hardware. These properties align with cyber monitoring, where data arrives asynchronously, and malicious behavior often emerges through temporal patterns across event sequences. However, cyber streams are not composed solely of continuous numeric signals: their informative structure is also carried by categorical identifiers, irregular timing, and local behavioral context. Traditional rate- and population-based spike encodings are not naturally suited to these heterogeneous semantics, while conventional intrusion detection system (IDS) pipelines typically resolve the mismatch by converting raw events into flows, fixed aggregation windows, or dense tensors. Although useful for conventional classifiers, these transformations introduce buffering latency, obscure native temporal structure, and weaken the computational advantages of event-driven neuromorphic processing. We introduce an event-native symbolic-temporal spike encoding framework that maps heterogeneous cyber events directly into sparse, spike-compatible inputs. By assigning encoding roles to semantic identity, local frequency context, and inter-event timing, the framework preserves categorical semantics and temporal dynamics. We validate the approach on packet-level Network IDS and extend it to message-level CAN IDS, using both domains to evaluate whether the encoding exposes usable structure for recurrent SNNs operating directly on native event streams. Under edge-oriented, $μ$Caspian-aligned hardware constraints, compact recurrent SNNs achieve strong anomaly detection performance, with an operational hybrid metric ($J_{hybrid}$) of 0.987 on Network IDS and 0.980 on CAN IDS.

0.34

Parameters efficient language model with latent cross-attention for IoT intrusion detection

cs.CR ·2026-10-21 ·Frederic Adjewa, Moez Esseghir, Leila Merghem-Boulahia, Cheikh Kacfah
similarity 0.66
Abstract · 14 pages

Language models have shown strong potential for intrusion detection, but their high memory and computational demands make them difficult to deploy in resource-constrained IoT environments. In this work, we propose a compact Transformer-based architecture that compresses the input into a small latent space before processing it with a minimal Transformer core. We analyze nine compression configurations, compare parameter sharing with layer stacking, and evaluate our approach against SecurityBERT on two IoT intrusion detection datasets, Edge-IIoTset and CICIoT2023. With only 306k non-embedding parameters and 1.6M parameters in total (6MB), our model is about 7x smaller than SecurityBERT (11.2M parameters, 45MB). With the same privacy-preserving encoding, it outperforms SecurityBERT by about 2 points of macro F1-score on Edge-IIoTset and is within 0.7 points on CICIoT2023. It is also about 89x faster on a simulated edge device. We further show that a single Transformer layer is sufficient, and that parameter sharing matches layer stacking with up to 66% fewer parameters. These findings show that lightweight Transformer-based architectures can provide a practical and accurate solution for intrusion detection in resource-constrained IoT settings

0.23

Recording-Aware Evaluation of Multimodal OT Intrusion Detection

cs.CR cs.LG ·2026-10-16 ·Mustafa Umut Ozbek, Khalil El-Khatib, Li Yang
publishedbenchmarkcode/data similarity 0.65
Abstract · 12 pages, 3 figures, 5 tables. Accepted for presentation at the Industrial Control System Security Workshop (ICSS 2026), held in conjunction…

Machine-learning-based intrusion detection systems (IDSs) for operational technology (OT) are commonly evaluated on benchmark datasets in which normal and attack data are collected in separate recordings. In such settings, high performance may reflect recording-associated differences as well as attack behavior. We introduce a recording-aware evaluation procedure that compares attack windows with background from the same recording, uses prior data for model fitting and calibration, tests temporal alignment against randomized event schedules, and reports event coverage together with alarm burden. On the ProvICS multimodal dataset, cyber modalities strongly distinguish the benign and attack recordings before the first labeled attack. Same-recording evaluation reverses the apparent modality ranking across all studied window sizes: the physical view becomes the strongest individual modality, while cyber-only fusion shows the largest difference between cross-recording and same-recording AUC, reaching 0.54. Operational analysis further shows that permissive event-overlap metrics can overstate detection when a small number of long alarm episodes cover multiple actions. An external analysis on HAI shows smaller and reference-dependent recording sensitivity. These findings show that cross-recording benchmark performance should be interpreted alongside same-recording, forward, temporal, and operational evaluations.

0.44

Temporal transformer CAN encoder with federated lightweight heads for anomaly detection

cs.LG cs.CR ·2026-10-07 ·Konstantinos Gyftodimos, Kyriakos Chiotis, Elena Politi, George Dimitrakopoulos, Eirini Liotou
publishedcode/data similarity 0.64
Abstract

Modern vehicles rely on large numbers of Electronic Control Units (ECUs) that constantly exchange information over the Controller Area Network (CAN) bus. Due to the rapidity, structure, and repetition of this communication, even slight variations in timing, payload values, or message patterns can point to unusual activity. Whether due to errors, malfunctions, or deliberate interference, these anomalies are frequently subtle and challenging to identify with conventional methods that handle messages separately or rely on manually created rules. Motivated by this gap, we present a privacy-preserving framework for anomaly detection in in-vehicle networks, based on a Temporal Transformer CAN Encoder with Federated Lightweight Heads, to better capture these irregularities. The detection of subtle temporal and contextual anomalies is made possible by a lightweight Transformer encoder that learns how these signals evolve over time, while a federated learning mechanism enables several vehicles or ECUs to work together to improve a shared model without exchanging raw CAN data. This combination of federated learning and temporal sequence modeling provides robust anomaly detection performance while maintaining efficiency and privacy, according to experiments conducted on open-source datasets.

0.51

ST-GAT: A Leakage-Free Spatio-Temporal Graph Attention Framework for Robust Network Intrusion Detection

cs.CR cs.AIcs.LG ·2026-11-14 ·Nikhil Parkar, Sanskar Pandey, Siddhi Prajapati, Suganeshwari G, Syed Ibrahim
similarity 0.62
Abstract

Intrusion detection systems are a central defence in modern networks. Deep learning models report strong anomaly-detection results, but evaluations on standard benchmarks such as CSE-CIC-IDS-2018 are often inflated by two methodological problems: temporal leakage from overlapping sliding windows, and contamination from preprocessing fitted before the train/test split. Per-flow classifiers also ignore the spatio-temporal dependencies across sequences of network flows. We propose a Spatio-Temporal Graph Attention Network (ST-GAT) that represents traffic as a sequence of graph snapshots. It builds k-Nearest Neighbour (k-NN) edges within each window, applies multi-head Graph Attention Network (GAT) layers for spatial aggregation, and uses a Bidirectional Long Short-Term Memory (BiLSTM) network with temporal attention for sequence modelling. To prevent leakage, we use non-overlapping temporal windows (stride equal to window length) and fit all preprocessing on the training split only. We compare three class-imbalance strategies: inverse class-frequency weighting, dynamic balanced sampling, and few-shot metric prototyping. Across three random seeds, ST-GAT with inverse class weighting reached a Macro-F1 of 96.65% (+/- 0.87) and an accuracy of 99.14%. This is 14.3 percentage points higher in Macro-F1 than a per-flow Random Forest baseline (82.34%). Balanced sampling reduced Macro-F1 by 2.3 points (to 94.37%) relative to loss weighting. This suggests that aggressive resampling of small, noisy minority classes can hurt detection in heavily imbalanced traffic.

0.34

Out of Sync, Out of Sight: Phantom State Attacks against IIoT Intrusion Detection

cs.CR cs.AI ·2026-10-01 ·Sabrine Ennaji, Elhadj Benkhelifa, Nadia Kabachi
similarity 0.62
Abstract

Machine learning-based intrusion detection systems (IDS) are critical for securing Industrial Internet of Things (IIoT) environments. Most adversarial research against them perturbs the feature vector or the traffic that produces it, and depends on gradient access, repeated model queries, or a learned model of benign traffic. A smaller line of work reshapes packet timing without querying the detector, but makes malicious traffic mimic a learned model of benign timing. Across these approaches, one assumption of industrial monitoring pipelines has received little attention: temporal synchronization. An IDS reconstructs operational state by aggregating telemetry into sliding or tumbling windows, so its view depends not only on what is observed but on when each observation falls relative to a window boundary. We introduce the Phantom State Attack (PSA), which exploits that dependence under a passive, zero-query threat model. Rather than modifying packets, perturbing features, querying the classifier, or fitting any model of benign traffic, PSA injects bounded timing drift calibrated to the attack flow's own inter-arrival variability, moving observations across the nearest window boundary by the minimal shift needed. The IDS then reconstructs a phantom state that diverges from the true process state. We evaluate PSA on ToN-IoT and CIC IIoT 2025 (DataSense), against Random Forest, MLP and XGBoost, measuring detection degradation, synchronization distortion, stealth, and attacker cost. PSA degrades detection on flows carrying enough packets for window-boundary redistribution, and leaves others almost unchanged, so its effect is conditional. A query-based baseline reaches higher raw success but needs many queries per window, while PSA needs none. The results identify temporal aggregation as an attack surface reachable under weaker assumptions than prior evasion techniques.

0.18

AutoDP-LLM: Automating Data Pre-processing for Intrusion Detection Systems using Large Language Models

cs.CR cs.AI ·2026-10-04 ·Bao-Phong Nguyen, Gia-Khanh Pham, Thai-Duong Do, Mai Xuan Trang, Minh-Tuan Le +5 more
top venuebenchmark similarity 0.61
Abstract

The increasing complexity and scale of modern cyber-attacks demand intelligent and computationally efficient Intrusion Detection Systems (IDS). However, designing effective data pre-processing pipelines traditionally involves substantial trial-and-error effort and repeated evaluation of alternative configurations. For large, high-dimensional network traffic data, this process can create a significant computational burden. In this work, we propose AutoDP-LLM, an automated pre-processing framework designed to reduce manual pipeline development and computational overhead. Specifically, AutoDP-LLM leverages Large Language Models (LLMs) to autonomously generate and validate executable data pre-processing pipelines. The framework combines deterministic host-side planning with LLM-based specialist agents to formulate data-processing strategies, synthesize executable code, and adaptively determine retained feature sets using semantic reasoning and training-derived statistical evidence, without requiring a predefined feature budget. Focusing on multiclass intrusion detection, we evaluate AutoDP-LLM on the UNSW-NB15 and NSL-KDD benchmark datasets using multiple downstream classifiers. Comparative experiments against conventional feature-selection methods show that AutoDP-LLM achieves competitive detection performance while automating the generation of compact and executable pre-processing pipelines. Component-level ablation experiments further demonstrate the complementary contributions of the semantic and statistical feature-reduction components. The repeated generation, validation, execution, and assessment of candidate pipelines are amenable to parallel execution, highlighting the potential of scalable computing environments, including high-performance computing (HPC) systems, to support automated IDS pipeline development.

0.39