CS Paper Radar

Design and Evaluation of a Controlled Post-Alert Incident Orchestration and Response Subsystem Using a Rule Engine and a Local Large Language Model

Hoang-Lam Huynh, Quoc-Cuong Tang, Van-Tri Phan, Khuong Nguyen-An

cs.CR cs.SE ·submitted 2026-09-22 12:26 UTC · v1
published

Abstract

This paper presents a controlled post-alert incident orchestration and response subsystem for educational information systems. The architecture separates deterministic classification, contextual analysis, human approval, and technical execution. A Rule Engine determines severity and selects the playbook, while Static RAG and a local large language model provide advisory content under Validator, Guardrail, Output Sanitizer, and Safe Fallback controls. Experiments begin after simulated alerts are stored in Elasticsearch. The Rule Engine matched the predefined routing matrix in all 30 boundary cases. The Durable Queue completed 100 events without duplicate tasks, new failed tasks, or unintended firewall rules. An eight-alert contention experiment preserved the configured limit of one active model request, and 30 sequential measurements showed an overall mean post-alert processing time of approximately 33 seconds. The results demonstrate functional correctness, traceability, controlled recovery, and bounded model integration within the evaluated laboratory scope.

Comment
Accepted for publication in the Proceedings of the 19th Conference on Fundamental and Applied IT Research (FAIR'2026), Ho Chi Minh City, Vietnam, October 8-9, 2026

Closest papers

Nearest neighbours by meaning across the whole index.

From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage

cs.CR cs.AIcs.MA ·2026-10-07 ·Saimon Amanuel Tsegai, Alex Kantchelian, Danfeng, Yao, Peng Gao
benchmarkhighly cited similarity 0.61
Abstract · Preprint

Security operations centers (SOCs) must triage large volumes of alerts, most of which are benign, while missed attacks can remain uninvestigated. Tool-using large language model (LLM) agents can retrieve evidence during triage, but it remains unclear how reasoning strategies determine what to gather and when an investigation is sufficient to close an alert. We study five representative approaches spanning single-pass tool use, iterative retrieval, sampled investigations, self-review, and explicit verification. To support this study, we build ALERT-BENCH, an interactive benchmark that replays enterprise telemetry through a live SIEM and requires each system to retrieve evidence. Across 1,247 alerts from a multi-stage attack scenario, every approach missed at least 40.4% of attack-related alerts. Trace analysis shows that attack alerts are more likely to be dismissed when searches return no records, same-context review has negative net correction, and dismissal receives no consistently stronger investigation than escalation. Based on these findings, we further design AIDA (Adversarial Investigation and Dialectical Analysis), a multi-agent framework that requires an explicit proposed decision before independent challenge and stronger evidentiary requirements before dismissal. AIDA preserves investigation history in an append-only Investigation Ledger and keeps the challenge in a separate reasoning context. A separate Judge adjudicates the proposed decision and challenge against evidence, resolving the alert or requesting another round when evidence is missing. On the same alerts, AIDA achieves an F1 score of 0.958, compared with 0.371-0.744 for the studied approaches, and reduces the false-negative rate from 40.4% to 3.1% while escalating 18.4% of alerts to analysts. These results show that structuring evidence retrieval and decision review can substantially improve agentic SOC triage.

0.45

Enterprise Process Control Tower for IT-Led Improvement

cs.CE ·2026-09-19 ·Shunmukha Sagar Puppala
similarity 0.60
Abstract · 10 pages

Enterprise integration traffic in large Enterprise Resource Planning (ERP) systems move through Electronic Data Interchange (EDI) and the Intermediate Document (IDoc), and the operational tools that surround that traffic remain fragmented. Status records are written for a technical audience, exception handling is performed one document at a time, and process health is read manually from long lists. This work proposes an enterprise process control tower that lets an Information Technology (IT) team observe, diagnose, and improve business workflows from a single operational layer. The design links process health observation, semantic translation of technical status text, machine-learning diagnosis of exceptions with model explainability, grouped recovery actions, and key performance indicators inside one governance framework. The proposed framework is evaluated on a synthetic IDoc event dataset of 84,000 records spanning twelve months, of which 15,120 error records are used to train a five-class remediation-routing model that maps each exception to a standard recovery action. A rule-based router that reproduces current production heuristics serves as the primary baseline. The proposed gradient-boosted diagnosis engine reaches a macro-averaged F1 score of 0.883 on the held-out test partition, against 0.742 for the rule-based router, while Shapely-based attributions expose the drivers of each routing decision to operators. Results are derived from synthetic data and are presented as an internal-consistency demonstration rather than real-world validation. The contribution is an integrated tool design that places diagnosis and governance on the same plane as observation.

0.28

Model-Based Authoring of SOC Simulation Exercises for Cybersecurity Education

cs.CR ·2026-11-10 ·Felix Härer
published similarity 0.58
Abstract · Presented at 2026 6th Asia Education Technology Symposium (AETS 2026), November 2026. Accepted for publication in the conference proceedings…

Cybersecurity training and classroom exercises with a virtual Security Operations Center (SOC) combine alerts, evidence, timeline entries, learner tasks, and assessment items that must remain consistent across an evolving incident. This paper presents a model-based authoring method in which a technical and educational specification is used to construct an incident exercise model and a large language model (LLM) generates selected scenario text based on that model. The incident exercise model combines an incident model of events, objects, and evidence with an exercise orchestration model of SOC alerts, timeline entries, learner tasks, and assessment items. The method is implemented in a Virtual SOC prototype and evaluated using a phishing-to-ransomware reference exercise. Its incident exercise model contains eight incident events, eight incident objects, six evidence items, seven alerts in five batches, 13 timeline entries including 12 learner tasks, and five representative assessment items. GLM-5.3-generated alert descriptions are inserted and a separate validator checks references, ordering, evidence links, assessment completeness, and selected fixed fields. Validation is demonstrated with reference data, resulting in true-negative outcomes for valid data and true-positive outcomes for deliberately placed inconsistencies. The resulting exercise package is executable within the Virtual SOC learner environment. Overall, the evaluation indicates consistency across the involved model elements and establishes the feasibility of the model-based and generative approach.

0.44

REFINE: A Resilient Evolution Framework for Intelligent Enterprise Alert Triage in Security Operations Centers

cs.CR cs.AIcs.LG ·2026-09-26 ·Huimin Chen, Quan Long, Yanhao Wang
similarity 0.57
Abstract

Security Operations Centers (SOCs) process large volumes of alerts daily. Alert triage prioritizes high-risk threats while reducing manual review of benign alerts. LLM agents can reason over logs and threat intelligence, but struggle to keep aligned with organization-specific, rapidly evolving SOC operational standards. We introduce REFINE, an LLM-agent framework for enterprise alert triage. REFINE encodes analyst expertise as structured skills and continuously adapts using analyst disposition feedback. It enforces recall = 1.0 as a hard constraint during evolution to maximize auto-closure of false positives, and identifies judgment blind spots by combining alert distributions with model error boundaries. Evaluated on four real industrial SOC scenarios across four MITRE ATT&CK phases with temporal split: REFINE achieves recall=1.0 on all evolution sets. On future test windows, it retains recall=1.0 in three scenarios; the degraded case reaches 0.807 recall, still outperforming self-evolution baselines (0.49-0.58).

0.22

STAIR: Effective Incident Response Using an End-to-End Agentic Planning Framework

cs.CR cs.AI ·2026-08-10 ·Hanlin Jiang, Jionghao Huang, Shaofei Li, Bojia Yu, Peng Jiang +4 more
benchmark similarity 0.55
Abstract · 21 pages, 4 figures

Incident response planning is critical for restoring compromised software systems. Existing AIOps methods automate fault diagnosis and remediation and adapt response using feedback, but lack a unified decision basis across planning and execution. This limits their ability to maintain incident state and align actions with evolving recovery objectives during long-horizon, multi-stage security incident response. We present STAIR, an end-to-end agentic response framework that organizes planning and execution around a shared incident state. STAIR maintains incident evidence, response actions, and execution feedback in Graph-as-State (GAS), using the latest state to determine the response stage and recovery objectives for stage-specialized planning. The framework retrieves effect-validated historical experience based on the current state and stage to guide action selection. An Execution Harness executes actions, returns feedback to update GAS, and validates action effects for future experience reuse. We construct STAIR-Bench, an executable incident response benchmark comprising 44 Docker-based cyber-range cases. On its held-out test cases, STAIR achieves an overall defense score of 0.95, improving over the strongest baseline by approximately 10.47%. In the independent CAGE4 environment with ongoing attacks and benign activity, STAIR reduces cumulative penalties by approximately 28.8% relative to the strongest baseline.

0.43

AeroCopilotBench: Safety-Gated Evaluation of LLM Agents on Aircraft Emergency Procedures in an Executable Cockpit

cs.AI ·2026-08-17 ·Yuchen Yuan, Zhenghuang Wu, Yuangan Li, Liang Ma, Ke Li
benchmark similarity 0.55
Abstract

Aviation knowledge question answering cannot directly assess the operational effectiveness and safety compliance of large language models throughout aircraft emergency procedures. We introduce AeroCopilotBench and its executable cockpit environment, ACOE, which define state-transition rules, task goals, and trajectory-level safety constraints based on aircraft-specific Pilot's Operating Handbooks (POHs). The benchmark comprises 12 scenario templates and 73 tasks across two aircraft types, evaluating task completion, safety compliance, execution discipline, and repeatability. Across repeated evaluations of 12 models, the highest safety-gated success rate is 72.6\%. Most failed episodes achieve all critical goals but do not satisfy all remaining terminal goals. Across repeated runs, models still omit steps that they execute in other runs of the same task. Analysis of failed trajectories further reveals that some reasons for actions that conflict with the aircraft's POH recur across models. These results expose shortcomings in complete procedure execution, consistency across runs, and aircraft-specific emergency response.

0.30

RAVEN: A Self-RAG Based Agent for Threat Classification and Response

cs.CR ·2026-11-19 ·Pironti Francesco Aurelio, Arena Luigi, Blefari Francesco, Lupinacci Matteo, Romeo Francesco +1 more
similarity 0.55
Abstract

Security event analysis in large enterprises can be overwhelming, as each alert typically requires manual in- spection by human operators. This process is time-consuming and prone to inconsistencies, especially in high-volume Secu- rity Operations Center (SOC) environments. In this work- in-progress paper, we present RAVEN: Retrieval-Augmented Vulnerability & Event aNalysis, an agentic framework designed to support SOC analysts by automating the initial stages of alert investigation. The proposed system leverages a Self- Reflective Retrieval-Augmented Generation-based approach to provide an initial classification of security events and enables interactive refinement through analyst-in-the-loop queries. Un- like traditional approaches that rely on generic mitigation strategies, RAVEN generates context-aware remediation sug- gestions by incorporating knowledge of the target enterprise infrastructure. We implement an early-stage prototype and report preliminary results on three balanced test sets of 400 curated web payloads each (SQLi, XSS, SSTI) and a qualitative case study of mitigation generation; an evaluation on IDS- generated alerts and a user study are ongoing and outlined as remaining work.

0.12

SAFESHIELD: A Decision-Organization Framework for Deployment-Time Safety of Small Language Models

cs.SE cs.AIcs.CR ·2026-10-05 ·Xingru Zhou, Luis Sentis, Aarti Choudhary
published similarity 0.55
Abstract · Accepted to the Application Track of IEEE TPS 2026. 12 pages

Deployment-time safety of language models is commonly implemented through runtime guardrails such as input moderation, routing, retrieval verification, and output filtering. Existing deployment frameworks provide increasingly capable mechanisms for these functions, but offer limited guidance on how the safety decisions they produce should be explicitly organized, coordinated, and audited. We formulate deployment-time safety as a decision-organization problem with two elements: responsibility-oriented decomposition of safety decisions and explicit coordination among them. We instantiate this formulation in SAFESHIELD, a deployment-time safety system for small language models that organizes four recurring decision responsibilities (admission, routing, evidence, and release) and records committed decisions in auditable Decision Traces. We evaluate SAFESHIELD through mechanism-level experiments, aggregate stage ablations, controlled coordination ablations, and a deployment-oriented stress suite. Mechanism-level results show that the instantiated safeguards provide the capabilities required by the decision process, while aggregate ablations show substantial degradation in end-to-end safety as the surrounding safety organization is removed. More importantly, dedicated coordination ablations preserve the participating safeguard mechanisms while selectively severing their dependencies: removing admission gating substantially increases false release, and withholding upstream evidence from the release decision reduces release accuracy from 96.0% to 69.5%. These results provide system-level evidence that deployment-time safety depends not only on the capability of individual guardrails, but also on how their decisions are organized and coordinated.

0.32