CS Paper Radar

The Cybersecurity Dataset Observatory: Which Data for AI in Cybersecurity?

Francesco Cerasuolo, Gabriele Mangiacapre, Antonio Pescapè

cs.CR cs.AI ·submitted 2026-11-16 09:58 UTC · v1
benchmarkdatasetcode/data

Abstract

AI is increasingly used for cybersecurity tasks such as intrusion and malware detection, in enterprise, industrial, vehicular and mobile networks. Its effectiveness depends on data: models need representative network traffic, both to be trained and to be evaluated realistically. Public datasets of cybersecurity network traffic, however, are scattered across laboratory pages, repositories and aggregators, and are continuously released, corrected and withdrawn: 92 of the 234 datasets we catalog were released in 2023 or later. At this pace, surveys that list them are outdated soon after publication, since they describe the field only as it was when they were written. We present the Cybersecurity dataset observatory, a public and continuously updated catalog of cybersecurity datasets based on network traffic. New datasets are added as they are found or submitted, every record is periodically re-checked against its original source, to which it links rather than hosting the data, and every change is recorded in a public change log. This paper documents the first release: we define a traffic-centered scope and a seven-axis taxonomy, and review 234 datasets (204 original captures and 30 derivatives) across 14 domains. The review shows a field that grows quickly but unevenly, where 91 datasets distribute only processed records, a few laboratories produce a large share of the data, and UAV, wireless and V2X networks have little public evidence, while 5G has few real captures. We then relate the catalog to common uses in AI research, outline a procedure for selecting datasets, and discuss benchmark pitfalls and open gaps.

Closest papers

Nearest neighbours by meaning across the whole index.

ICS Cybersecurity Datasets: A Systematic Meta-Review of Coverage, Evaluation Practice, and Structural Gaps

cs.CR ·2026-08-25 ·Konstantinos E. Kampourakis, Vyron Kampourakis, Georgios Kambourakis, Sokratis Katsikas, Stefanos Gritzalis +2 more
similarity 0.72
Abstract

Intrusion detection research in Industrial Control Systems (ICS) heavily depends on public datasets, yet no prior work has systematically assessed whether the collective dataset corpus supports current evaluation claims. This paper addresses this gap through a meta-review of 18 studies between 2019 and 2026, from which 83 ICS, or ICS directly related, cybersecurity datasets are identified, harmonised, and characterised using a unified five-dimensional taxonomy. The taxonomy reveals that the corpus is structurally skewed: 85.5% of datasets concentrate on late-stage OT Disruption tactics, cross-stage IT/OT progression sequences are present in only 8.4% of cases, field-device evidence at Level 0 of the Purdue hierarchy is effectively absent, and operationally sourced data accounts for only 15.7% of the collection. A parallel audit of evaluation practices shows that zero report streaming evaluation, fewer than half apply disciplined train/test partitioning, and only two satisfy reproducibility requirements. Furthermore, a taxonomy-evaluation coupling analysis shows that dataset imbalances constrain the scope and feasibility of several evaluation practices. Based on these findings, we identify three structural imbalances: i) architectural shallowness, ii) progression compression, and iii) cross-domain substitution, and derive a coordinated research agenda which covers cross-stage corpus construction, temporally structured benchmarking, event-level label standards, and governance frameworks for operational data sharing.

0.42

SecLog-Bench: A Scalable Benchmark for Cybersecurity Log Reasoning with Scaffolded Evidence

cs.CR cs.AI ·2026-10-22 ·Yifan Wang, Dongliang Xu, Xiaoyong Li, Jing Li, Peter S. Pan +1 more
benchmarkcode/data similarity 0.65
Abstract · 17 pages, 9 figures. Submitted to IEEE Transactions on Dependable and Secure Computing. Code and data: https://github.com/SecLog-Open/SecLog

Large language models (LLMs) have shown strong potential for cybersecurity, leading to growing interest in LLM-based automated cybersecurity log analysis. However, current datasets (1) are limited in scale; (2) rely mainly on single-source evidence; and (3) have only binary or multi-class attack labels, without dense annotations of the reasoning process. To address these limitations, SecLog-Bench has three key features. First, we build an extensible simulation platform that covers 29 attack scenarios, supports repeated execution and the integration of new scenarios, and enables scalable generation of cybersecurity logs with attack-stage labels. Second, we use multi-source logs, including audit logs, application logs, and network traffic, to jointly analyze cyber attacks. Third, we annotate each sample with the essential context for cybersecurity log reasoning and a structured step-by-step evidence chain. We evaluate eight mainstream LLMs on this benchmark for post-detection attack-progress reasoning. Results show that, when provided with scaffolded evidence, current LLMs exhibit modest capability in attack-progress reasoning, with the best model achieving only 44.44% accuracy and 45.07% Macro-F1. Their reasoning remains unreliable; i.e., the highest Stage Reasoning F1 (SRF1) score is only 54.87%. Moreover, even correct predictions may be accompanied by flawed reasoning, and reasoning quality is often misaligned with prediction accuracy.

0.35

A Roadmap to Available ICS Datasets and Testbeds for Cybersecurity Research

cs.CR eess.SY ·2026-08-31 ·Ebtesam J. Alqahtani, Mohammad Hammoudeh
similarity 0.63
Abstract

Industrial Control Systems (ICS) are the backbone of many critical infrastructure sectors; however, their growing level of connectivity, long lifespan and integration with the Information Technology (IT) environment introduces numerous cybersecurity challenges. The merging of Operational Technology (OT) and IT along with the deployment of Industry 4.0 technologies increases the attack surface of ICS environments, which in turn makes them more vulnerable to advanced cyber threats. Therefore, many researchers have shown interest in the field of cybersecurity of ICS. The topics of intrusion detection, anomaly detection, threat intelligence, attack simulation and resilience assessment of ICS have received much attention. Nevertheless, the development and testing of cybersecurity solutions for ICS remains to be challenging due to the lack of appropriate datasets and experimental environment. The main objective of this paper is to provide the roadmap of existing ICS cybersecurity datasets, testbeds and digital twins. This paper presents various taxonomies along with systematic analysis of architecture, characteristics, capabilities, pros and cons of these tools. The results of the analysis demonstrate the presence of persistent problems such as lack of standardized benchmarking datasets, lack of modern attack scenarios, insufficient number of datasets based on real operational traffic and difficulty in validating artificial intelligence-driven cybersecurity solutions. In addition to summarizing current research on ICS cybersecurity datasets and testbeds, this roadmap provides the identification of research gaps and recommendations on creation of new tools.

0.20

ISIA-AF: Orchestrating Reproducible Attacks and Multi-Source Data Collection for OT Systems

cs.CR ·2026-09-16 ·Stefan Manfred Haratzmüller, Thomas Rosenstatter, Olaf Saßnick, Dalibor Sain, Stefan Huber
published similarity 0.62
Abstract · 8 pages, 1 Figure, 1 Table, submitted version to Euromicro Software Engineering and Advanced Applications (SEAA)

Operational Technology (OT) environments require realistic, reproducible security datasets, yet existing approaches often lack automation, multi-source data capture, and sufficient documentation for reuse. This paper presents ISIA-AF, a modular attack framework for orchestrating reproducible attack execution and automated dataset generation on industrial systems. The framework coordinates distributed attack clients, records network traffic and operational data, ultimately leading to a multi-source dataset. We derive functional and non-functional requirements from prior work and stakeholder discussions, and realise the framework following a design science research approach. A case study on the ISIA testbed, comprising a real industrial system and a simulated process, demonstrates how the framework supports centralised control, low communication overhead, and flexible deployment across network segments. The result is a practical basis for generating extensible, multi-source OT security datasets for intrusion detection research.

0.37

Autonomous Cyber Operations Gym for Training and Evaluating AI Models for Defence

cs.CR cs.LG ·2026-10-13 ·Tobias Hürten, Johannes F. Loevenich, Florian Spelter, Erik Adler, Laurin Holz +3 more
published similarity 0.62
Abstract · 6 pages, 5 figures, 1 table. Accepted and presented at the 2026 IEEE International Conference on Cyber Security and Resilience

This paper proposes an Autonomous Cyber Operations (ACO) gym, a unified framework integrating high-fidelity network emulation, realistic user behaviour modelling for benign network traffic and adversarial profiles, and the generation of cybersecurity scenarios through open toolkits. By supporting live and deferred feature extraction at the packet and flow levels, the gym facilitates the creation of labelled synthetic datasets for training Deep Neural Network (DNN)-based Intrusion Detection Systems (IDSs), as well as developing and evaluating modular components for autonomous defence agents through emulated interactions. While evaluation of a DNN classifier within the gym shows high in-domain performance, achieving near-perfect recall on adversarial network flows, our experiments reveal a significant sensitivity to domain shift, showing that even minor changes in network composition, such as variations in traffic protocol distribution, can lead to a catastrophic collapse in model transferability and detection performance.

0.22

Generative AI for Encrypted Traffic Analysis: Synthetic Dataset Generation and Classifier Evaluation

cs.CR ·2026-08-10 ·Harshil Patel, Himanshu Garg, Aswani Kumar Cherukuri
similarity 0.62
Abstract

Network traffic analysis faces significant challenges with encrypted communications, primarily due to limited visibility into packet contents and the inherent imbalance in available datasets, particularly for anomalous traffic patterns. This paper addresses these challenges by exploring Generative AI (GAI) techniques to create realistic and balanced synthetic encrypted traffic datasets. Our approach incorporates feature analysis, clustering-based data generation, and comprehensive classifier evaluation to ensure synthetic data quality. We demonstrate that properly generated synthetic data can effectively supplement real- world datasets, achieving up to 93% performance when training classifiers compared to those trained on real data. The proposed methodology preserves critical statistical properties and feature correlations while enabling the creation of balanced datasets, ad- dressing the persistent challenge of anomaly underrepresentation in cybersecurity data. Along with the results we provide complete programming code designed and implemented in this work.

0.04

eBPF-Based Cybersecurity Mechanisms: A Systematic Literature Review

cs.CR ·2026-08-27 ·Stamatios Kostopoulos, Panagiotis Tsakonas, Evangelos K. Markakis
publishedsurvey similarity 0.62
Abstract · 24 pages, 6 figures, journal, systematic literature review, 8 tables, Accepted for publication in the International Journal of Information S…

Extended Berkeley Packet Filter (eBPF) has emerged as a kernel-level framework enabling dynamic security enforcement in modern operating systems. While eBPF's cybersecurity potential has attracted significant attention, existing work remains fragmented across domains, evaluation methodologies, and deployment contexts. This systematic literature review applies PRISMA methodology to identify, categorize, and synthesize peer-reviewed research on eBPF-based cybersecurity mechanisms. Following a structured screening of 3,735 records from six databases, 54 primary studies (2018-2026) were analyzed and organized into a seven-domain taxonomy: DDoS mitigation, intrusion detection, IoT security, container security, microservice protection, networking, and security tools. Analysis reveals eBPF enables low-overhead security enforcement (median 2.4% CPU overhead [1.1-8.6%], ranging from nanosecond-scale costs for infrequent hooks to 10-20% for kernel hot paths) with high detection accuracy (94-99%). It particularly excels in kernel-level monitoring, real-time packet processing, and cloud-native workload protection. However, significant challenges persist: verifier-imposed constraints limit algorithm complexity, 85.1% (46/54) of studies require low-level programming expertise, kernel version fragmentation hinders portability, and 96.2% (52/54) of research fails to address eBPF's own vulnerabilities. This review identifies critical research gaps in multi-tenant isolation, adversarial machine learning (ML) robustness, production validation, and standardized evaluation frameworks. By consolidating fragmented knowledge and highlighting architectural trade-offs between safety and expressiveness, this work provides a foundation for next-generation eBPF security systems and actionable directions for kernel programmability research.

0.62

Slips: Behavioral Evidence Aggregation for Network Security

cs.CR ·2026-08-12 ·Sebastian Garcia, Veronica Valeros, Alya Gomaa, Ondřej Lukáš, Martin Řepa +5 more
similarity 0.61
Abstract

Network intrusion detection systems often analyze individual packets or flows, although malicious behavior may develop across many connections and over time. This may limit their ability to combine isolated detections into a coherent assessment of host behavior. Packet-level features may also be too low-level for complex AI-based detection, requiring additional processing to improve accuracy while maintaining a low false-positive rate. We present Slips, a network intrusion detection system that builds host-centered behavioral profiles and organizes activity into time windows. It uses a modular architecture in which independent modules report evidence rather than generating final alerts directly. Slips then accumulates this evidence into host-level decisions. We evaluate Slips against Suricata on an expert-labeled PCAP dataset. At the profile-time-window level, Slips achieved 83% higher recall and a 70% higher F1 score than Suricata, while neither system produced false positives. These results indicate that time-window-based evidence accumulation can produce context-aware decisions that better align with expert judgment.

0.04