CS Paper Radar

How It's Made: Uncovering Detection Engineering Processes for Network Intrusion Detection Rules

Koen T. W. Teuwen, Emmanuele Zambon, Luca Allodi

cs.CR cs.HC ·submitted 2026-09-22 09:04 UTC · v1
published

Abstract

Many Security Operations Centers rely on signature-based Network Intrusion Detection Systems like Suricata, yet detection rule engineering remains understudied. We investigate this process by introducing SuriCap, a platform for rule engineering exercises, and hosting CTF-style workshops where 60 participants, trained MSc students, and experienced SOC professionals, created rules for four scenarios. Participants produced 3146 valid rules, enabling analysis of their methods, performance, and iteration patterns. Surprisingly, prior experience had limited impact on rule quality, suggesting that less experienced engineers can produce rules comparable to experts. We also observed challenges in generalizing rules beyond available tests, underscoring the need for sufficient labeled data. From our study, we identify three phases and a common pattern in rule engineering, offering SOC managers insights to improve their processes and expectations of engineer expertise.

DOI
10.1145/3846375.3849098

Closest papers

Nearest neighbours by meaning across the whole index.

RuleAutoPilot: Synthesizing Deployable Suricata Rules from Network Traffic

cs.CR ·2026-09-14 ·Mughees Ur Rehman, Aritran Piplai, Murat Kantarcioglu
similarity 0.71
Abstract

Rule-based Intrusion Detection Systems (IDS) such as Suricata are central to network security, yet crafting effective detection rules demands deep expert knowledge and cannot keep pace with emerging threats. Existing LLM-based approaches can reduce analyst effort, but they either rely on curated threat intelligence that is produced only after the underlying traffic artifacts already exist, or they require costly LLM use without sufficient quality control. We present RuleAutoPilot, an end-to-end agentic framework that generates deployable Suricata rules directly from malware network traffic, with no prior threat intelligence required. A key challenge is noise: network traffic captures often contain a small amount of security-relevant traffic mixed with large volumes of background traffic, which reduces LLM reasoning quality and increases cost. RuleAutoPilot addresses this challenge with a Benign Traffic Fingerprinting stage that removes known benign background flows before LLM processing. Rules that fail syntax checks, do not trigger on the source traffic, or generate false positives on a benign corpus are automatically repaired using structured feedback. Across 1,296 malware PCAPs, execution-grounded verification raises rule quality (F1) from 0.443 to 0.539. On a stratified 200-PCAP subset, RuleAutoPilot on the open-weight gpt-oss-120b reaches near-frontier quality, 0.524 F1 against Claude Opus 5 under Claude Code's 0.623, at 52x lower billed-token cost. Swapping only the backbone to Claude Opus 5, RuleAutoPilot surpasses Claude Code outright, 0.656 F1 against 0.623, at 40x fewer tokens. A stronger backbone raises RuleAutoPilot's own ceiling, but at the same backbone, our scaffold still outperforms Claude Code's, showing the scaffold contributes independently of the backbone.

0.13

Verification-Guided Specification Synthesis with Large Language Models for Intrusion Detection Rules

cs.CR ·2026-08-24 ·Kohei Yamamoto, Marie Katsurai
top venue similarity 0.65
Abstract · Accepted at CIKM 2026

Attacks against Internet-connected IoT devices continue to increase; however, transforming observed attack traffic into deployable intrusion detection system (IDS) rules remains largely a manual process. Recent studies have explored using large language models (LLMs) to generate IDS rules; nonetheless, existing approaches often require auxiliary information beyond observed traffic or generate rules without validating their detection logic against benign traffic. This study presents a verification-guided specification synthesis framework for generating Suricata rules directly from HTTP request traces. Instead of having an LLM generate IDS rules in a single step, an LLM first identifies a vulnerable parameter and synthesizes a semantic detection specification. These specifications are iteratively refined through counterexample-guided inductive synthesis (CEGIS), in which benign traffic samples serve as counterexamples during synthesis and verification. Verified specifications are then deterministically compiled into Suricata rules. Experiments on 281 real-world CVEs and benign traffic collected from real IoT devices show that the proposed method achieves a detection rate of 81.5% while maintaining a false positive rate of 0.0%. An ablation study also demonstrates that CEGIS-based verification improves detection performance while maintaining a low false positive rate.

0.58

Slips: Behavioral Evidence Aggregation for Network Security

cs.CR ·2026-08-12 ·Sebastian Garcia, Veronica Valeros, Alya Gomaa, Ondřej Lukáš, Martin Řepa +5 more
similarity 0.64
Abstract

Network intrusion detection systems often analyze individual packets or flows, although malicious behavior may develop across many connections and over time. This may limit their ability to combine isolated detections into a coherent assessment of host behavior. Packet-level features may also be too low-level for complex AI-based detection, requiring additional processing to improve accuracy while maintaining a low false-positive rate. We present Slips, a network intrusion detection system that builds host-centered behavioral profiles and organizes activity into time windows. It uses a modular architecture in which independent modules report evidence rather than generating final alerts directly. Slips then accumulates this evidence into host-level decisions. We evaluate Slips against Suricata on an expert-labeled PCAP dataset. At the profile-time-window level, Slips achieved 83% higher recall and a 70% higher F1 score than Suricata, while neither system produced false positives. These results indicate that time-window-based evidence accumulation can produce context-aware decisions that better align with expert judgment.

0.04

ACE: An AI-Orchestrated Intrusion Detection and Compliance Mapping Engine

cs.CR ·2026-10-15 ·Abdoulie Ceesay, Rishikesh Sahay, Md Rasel Al Mamun, Bell Raj Eapen
similarity 0.63
Abstract

This study proposes the Automated Compliance Engine (ACE), an AI-orchestrated framework that links intrusion detection, regulatory control mapping, and automated countermeasure deployment for security operations environments. ACE integrates Suricata IDS, a LangGraph five-stage workflow, retrieval-augmented generation, NIST SP 800-53 Rev. 5 control retrieval, large language model-based control ranking, and SSH-based Suricata rule deployment. The framework was evaluated in a two-virtual-machine laboratory using eight simulated attack scenarios and three legitimate traffic tests. ACE processed all eight attack alerts, achieved 100% alert detection, 0% false positives across legitimate traffic tests, generated deployable AUTO rules for 37.5% of alerts, verified 100% deployment of generated rules, and achieved an average NIST control-mapping confidence score of 0.87. The DeepEval assessment produced 1.0 scores for answer relevancy and faithfulness, while the hallucination score reflected known LLM-as-judge calibration limitations. The evaluation was conducted in a controlled environment with a limited attack set. Further validation is required using enterprise-scale telemetry, multiple IDS sensors, and an analyst-in-the-loop review. The results demonstrate how AI-assisted security automation can reduce analyst effort in translating IDS alerts into compliance-relevant evidence and defensive actions. ACE contributes an integrated detection-to-compliance-to-response workflow combining IDS telemetry, RAG-based control mapping, and automated rule deployment.

0.12

DeepWeave: Weaving Neural and Symbolic Evidence for ML-Based Network Intrusion Detection System

cs.CR cs.LGcs.NI ·2026-10-30 ·Raoul Saad, Enkeleda Bardhi, Andrea Agiollo, Fernando Kuipers
similarity 0.59
Abstract

Machine learning-based network intrusion detection systems can recognize complex traffic patterns, but often require substantial labelled attack data, degrade under deployment shift, and provide limited evidence for analysts. We present DeepWeave, a probabilistic neuro-symbolic Network Intrusion Detection System (NIDS) that makes logic an integral part of detection. DeepWeave automatically constructs and validates interpretable rules, represents neural predictions as probabilistic predicates, and reconciles both evidence sources through explicit coupling policies over packet-, flow-, and hybrid-views. The resulting proofs provide intrinsic explanations for rule-supported alerts. Across multiple datasets, DeepWeave improves low-data detection, false-positive control, transferability, and robustness. With only 50 labelled packet observations per attack class, it improves macro-F1 from 0.632 to 0.924 and reduces the attack false-positive rate from 0.075 to 0.008 over the matched neural baseline. Its proofs are also more stable than post-hoc neural attributions, while batched inference keeps probabilistic reasoning practical. These results demonstrate label-efficient, adaptable, and inspectable neuro-symbolic intrusion detection.

0.39

Network Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of Custody

cs.CR cs.LG ·2026-11-04 ·Jose Luis Vela Alonso, Carmen Pellicer
similarity 0.57
Abstract · 25 pages, 8 figures, 5 tables. Companion paper to arXiv:2607.00763

Deploying an explainable intrusion detection model from a validated laboratory setting into a forensic workflow requires guarantees that benchmarks do not test: that the certified model is the exact artifact applied to a case, that heterogeneous evidence can be classified reliably, and that every step is verifiable. We present MODEXRE, a forensic pipeline that extends a previously validated synthetic-data-trained XGBoost classifier with SHAP explainability (arXiv:2607.00763) into a dual-mode architecture: a Laboratory mode, where models are trained and certified on public and synthetic data, and a Formal mode, where a frozen, hash-verified model performs inference on case evidence under a SHA-256 hash-chained chain of custody. Suricata alerts, CEF firewall logs and packet captures are normalized through OCSF. Four design decisions, each triggered by an observed failure, shaped the architecture: a timestamp leak that inflated Kitsune performance (F1-macro 0.984, 0.662 after correction); a feature-coverage gap that made the 78-feature reference model misclassify a real SSH alert; causal temporal flow aggregation, which raised Kitsune multiclass F1-macro from 0.394 to 0.932; and a certification defect, exposed only by a real packet capture, whereby the pipeline had always trained a binary classifier. After correction, the same capture went from 100% of events flagged as malicious to 87% classified as Normal, with the scanning host identified as Reconnaissance. The default four-class certified model reaches F1-macro = 0.9538 on CICIDS2017-derived evidence; the recertified 18-category model reaches 0.6214, reflecting the real multiclass task. Chain-of-custody verification, model integrity hashing and per-prediction SHAP explanations are reported as measured outcomes.

0.17

VPID: An Integrated Framework for Vulnerability Prioritization and Intrusion Detection in Enterprise Networks

cs.CE ·2026-09-01 ·Xuanren Chen, Xin Wang, Xiaoqi Li
similarity 0.57
Abstract · 21 pages, 6 figures, 8 tables

Small enterprises face increasingly serious threats to their internal networks but often lack the financial resources, computing capacity, and specialist staff required to deploy resource intensive security platforms. This paper designs and implements VPID, a lightweight framework for vulnerability prioritization and intrusion detection that consists of two principal modules: controlled vulnerability validation and intelligent intrusion defense. The first module uses OpenVAS for asset mapping and vulnerability identification, applies a decision tree to prioritize vulnerabilities, and employs a rule engine to generate targeted validation payloads. The second module captures network traffic using Scapy, analyzes it through a detection pipeline that combines a decision tree with multinomial Naive Bayes, verifies traffic assessed as high risk using Snort rules, and performs blocking and alerting through iptables. The evaluation uses 550,000 network flow samples containing normal and attack traffic for detector training, together with 15,000 labeled vulnerability records. On the vulnerability ranking test set, the decision tree achieves a precision of 91.8%, a recall of 89.5%, and an F1 score of 90.6%. On an independent test set containing 55,000 traffic samples, the combined detection pipeline achieves a precision of 94.5%, a recall of 88.3%, and an F1 score of 91.3%, while maintaining a false positive rate below 1.5%.

0.13

IDSForge: Self-Explaining Network Intrusion Detection Through Neural Behavioral Rule Learning

cs.CR ·2026-10-28 ·Hao Fu, Jinxia Wei, Wan Wei, Chun Long, Degang Sun +1 more
similarity 0.56
Abstract

Anomaly-based network intrusion detection systems (NIDS) can detect previously unseen attacks by identifying deviations from benign traffic. However, deep learning-based anomaly detectors typically rely on implicit decision boundaries, and post-hoc explanations may not faithfully reflect the detection logic. This makes alerts difficult to interpret and audit. To address this issue, we present IDSForge, an anomaly-based NIDS that directly learns explicit, executable behavioral rules from benign traffic, making its detection logic interpretable by design. IDSForge uses the same rule set for detection and explanation, ensuring explanation fidelity by construction. It uses an explicit distributional autoencoder trained only on benign samples to learn axis-aligned regions of benign traffic. A high-speed rule compiler then consolidates these instance-specific regions into a compact set of feature-level interval constraints. These rules can be executed directly by popular NIDS engines, supporting high-throughput inspection with low overhead. To adapt to changes in traffic, a self-distillation co-evolver updates the model and rule set incrementally when the false-positive rate on analyst-verified traffic exceeds a target threshold. Our evaluation shows that IDSForge reduces F1 error by $5.7\times$ to $47\times$ relative to state-of-the-art anomaly-based baselines. Deployed in Suricata, IDSForge sustains a throughput of 7.8 Gbps with a median per-packet latency overhead of 3.1 $μ\text{s}$.

0.23