CS Paper Radar

Constraint-Aware Evaluation of Reinforcement Learning and Unsupervised Anomaly Detection for Security Operations Alert Triage

Radityo Ghifiardi, Rifny Rachman

cs.CR ·submitted 2026-11-03 23:35 UTC · v1

Abstract

Security Operations Centers (SOCs) must identify real threats from large alert streams, but limited analyst capacity means that unnecessary escalations can delay more serious investigations. Existing studies often frame SOC alert triage as classification or anomaly detection, but both views are incomplete for production SOCs, where the central problem is selecting alerts worth escalating under limited capacity, asymmetric error costs, and workload constraints. This paper addresses that gap by formalizing alert triage as a constrained decision problem and evaluating three approaches: an offline reinforcement learning policy trained on alert embeddings, an Isolation Forest model built on operational features, and an autoencoder trained on historical benign alerts. These methods represent different triage assumptions: learning actions from reward, isolating unusual operational patterns, and detecting deviations from benign behavior. We evaluate all methods on 74,909 analyst-labeled production alerts under a strict temporal split and benign-escalation constraint. Because the test set contains 503 unique feature patterns, we primarily rank clusters of similar alerts. At the main operating point, all three methods satisfy the constraint, with the reinforcement learning policy achieving the lowest benign-escalation fraction at 0.000, compared with 0.007 for the autoencoder and 0.041 for Isolation Forest. The methods also prioritize largely different alert clusters, showing complementary rather than redundant coverage. These results suggest that production SOC triage can benefit from constraint-aware evaluation and combined or adaptive deployment of different model families.

Closest papers

Nearest neighbours by meaning across the whole index.

Can Risk-Based Alerting Mitigate Cybersecurity Alert Fatigue?

cs.CR ·2026-09-02 ·Rafael Uetz, Philipp Bönninghausen, Louis Hackländer-Jansen, Martin Henze
similarity 0.73
Abstract · Submitted to USENIX Security '27

Security operations centers (SOCs) face large numbers of false alerts, making detection of cyberattacks difficult under typical resource constraints. Risk-based alerting (RBA) has been proposed as a means to reduce false alerts and has reportedly succeeded in doing so in various enterprise deployments. However, RBA has not been comprehensively evaluated until now, leaving implementation mostly guesswork based on anecdotal evidence. In this paper, we present the first systematic evaluation of RBA. To this end, we reformulate it as a continuous alert prioritization problem rather than a binary decision problem (i.e., whether an alerting threshold is exceeded), allowing us to evaluate performance across all possible thresholds and thus model SOCs of varying sizes and alert volumes. We distill five fundamental risk hypotheses, formalize them as independently parametrizable modules, and implement them in our novel experimentation suite CATS. We thoroughly assess the hypotheses across eight diverse alert datasets, six of which we created or extended to make such an evaluation possible. Our results show that certain combinations of hypotheses achieve a remarkable alert prioritization performance (AUROC $μ=0.92$, $σ=0.09$ across the eight datasets), outperforming a straightforward prioritization by alert severity level (AUROC $μ=0.72$, $σ=0.21$). We conclude that RBA can substantially reduce the number of false alerts that analysts have to review and thus has the potential to mitigate cybersecurity alert fatigue. In addition, it serves as a strong baseline for more complex, resource-intensive alert triage approaches (e.g., based on large language models).

0.17

REFINE: A Resilient Evolution Framework for Intelligent Enterprise Alert Triage in Security Operations Centers

cs.CR cs.AIcs.LG ·2026-09-26 ·Huimin Chen, Quan Long, Yanhao Wang
similarity 0.67
Abstract

Security Operations Centers (SOCs) process large volumes of alerts daily. Alert triage prioritizes high-risk threats while reducing manual review of benign alerts. LLM agents can reason over logs and threat intelligence, but struggle to keep aligned with organization-specific, rapidly evolving SOC operational standards. We introduce REFINE, an LLM-agent framework for enterprise alert triage. REFINE encodes analyst expertise as structured skills and continuously adapts using analyst disposition feedback. It enforces recall = 1.0 as a hard constraint during evolution to maximize auto-closure of false positives, and identifies judgment blind spots by combining alert distributions with model error boundaries. Evaluated on four real industrial SOC scenarios across four MITRE ATT&CK phases with temporal split: REFINE achieves recall=1.0 on all evolution sets. On future test windows, it retains recall=1.0 in three scenarios; the degraded case reaches 0.807 recall, still outperforming self-evolution baselines (0.49-0.58).

0.22

Constrained-Action AI Remediation for SIEM/XDR via a NeMo-Guardrails Proxy

cs.CR cs.AIcs.CL ·2026-10-07 ·Georgios Koutidis, Nikolaos Kekatos, Tom Nianios, Alexios Lekidis
published similarity 0.65
Abstract · 7 pages, 3 figures, 4 tables. Accepted at the 2026 IEEE International Conference on Cyber Security and Resilience (IEEE CSR 2026)

Security Operations Centers (SOCs) for information technology and operational technology share one incident-response problem: a flood of correlated alerts and too few analysts. Large Language Models (LLMs) are increasingly proposed as reasoning engines that triage alerts and, in autonomous deployments, issue commands that block IPs, kill processes, or quarantine files on production hosts. This coupling introduces a new risk: a single adversarial alert can become a remote code path through the LLM's reasoning, leading it to recommend an action the SOC then executes. We present a constrained-action architecture with two coordinated layers: (i) a SIEM/XDR control plane that grounds remediation in correlated host events and confines the LLM's output to a closed intent vocabulary whose templated commands are executed by thin endpoint agents, backstopped by an argument validator; and (ii) a NeMo-Guardrails proxy that wraps the SOC-analyst LLM with input- and output-rail policies, evaluated out-of-the-box against a SOC-specific adversarial corpus we release. The stock proxy lifts injection recall from 25.0% to 94.5% at a 0.1% false-positive rate, and a live red-team exercise confirms that the closed intent vocabulary and argument validator contain the observed LLM failure modes before any command crosses the trust boundary. As an architectural fit (not yet a measured operational-technology deployment), the constrained-action property suits critical-infrastructure settings where a wrong remediation has physical, not merely operational, consequences. The loop is best run human-in-the-loop or delayed: the measured rail latency keeps inline control out of scope.

0.58

XION: Interpreting Network Traffic Anomalies via Dependency Violations

cs.CR ·2026-08-27 ·Federica Uccello, Simin Nadjm-Tehrani
similarity 0.65
Abstract

Current research on security monitoring is increasingly focusing on machine-learning-based approaches, but caveats remain. In addition to huge computational overhead, one concern is the lack of insights into ``why" alerts are raised. Existing interpretability approaches rely on feature attribution methods that ignore dependencies among features or on causal modeling that requires extensive domain knowledge or computational resources. This work proposes XION, a method for modeling relationships among network-flow features based on benign traffic only. During detection, anomalies are identified through violations of expected feature dependencies. Further, XION supports post-alert analysis by identifying which feature relationships break, when they break along the attack timeline, and how dependency violations evolve relatively to other identified violations. XION is evaluated on standard IDS datasets and compared against an Isolation Forest (IF) baseline across multiple attack scenarios, including both volumetric and stealthier attacks. Results show that XION matches or exceeds IF recall in all evaluated scenarios, while requiring up to 7$\times$ less inference time. At the post-alert stage, the dependency violation analysis reveals temporal and structural patterns consistent with known attack behaviors, and is further evaluated against SHAP-based feature attribution of IF alerts. Together, these findings confirm that attacks indeed disrupt feature dependencies learned from benign traffic, and that these disruptions provide additional information for understanding an alert.

0.22

Integrating Perception, Comprehension, and Projection for Alert Validation: A Unified Cyber Situational Awareness Framework

cs.CR ·2026-10-29 ·Navodika Karunasingha, Mohan Baruwal Chhetri, Surya Nepal, Cecile Paris, Salil S. Kanhere
similarity 0.64
Abstract · 17 pages

Security Operation Centres (SOCs) must validate large volumes of alerts with limited analyst resources, yet alerts often lack sufficient context to determine whether suspicious activity is attack related. While provenance-based intrusion detection research has progressed from perception-level signals to comprehension- and projection-level cyber situational awareness (CSA), it remains unclear whether richer CSA consistently improves alert validation. This paper presents the first systematic evaluation of perception-, comprehension-,and projection-level CSA within a unified alert validation framework. Using Captain and Kairos, we generate multi-level CSA and formulate alert validation as a positive-unlabelled multi-instance learning (PU-MIL) task over CSA-derived behavioural hypotheses. Experimental evaluation on the DARPA TC dataset shows that comprehension-level CSA substantially improves validation by incorporating causal attack context. Relative to the Queue and Max. CSG baselines, PU-MIL achieves up to 28.6 and 75 percentage-point gains in Recall@5, and 24.6% and 86.8% reductions in mean positive rank on Cadets and Theia, respectively. Projection-level CSA can yield additional benefits, reducing the mean positive rank on Theia from 22.0 to 8.5 at a 40-minute projection horizon, although its effectiveness varies across projection horizons. Overall, richer CSA does not necessarily lead to better alert validation. Rather, its value depends on when and how it is introduced, motivating adaptive CSA progression that selectively enriches alerts according to validation requirements.

0.25

From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage

cs.CR cs.AIcs.MA ·2026-10-07 ·Saimon Amanuel Tsegai, Alex Kantchelian, Danfeng, Yao, Peng Gao
benchmarkhighly cited similarity 0.63
Abstract · Preprint

Security operations centers (SOCs) must triage large volumes of alerts, most of which are benign, while missed attacks can remain uninvestigated. Tool-using large language model (LLM) agents can retrieve evidence during triage, but it remains unclear how reasoning strategies determine what to gather and when an investigation is sufficient to close an alert. We study five representative approaches spanning single-pass tool use, iterative retrieval, sampled investigations, self-review, and explicit verification. To support this study, we build ALERT-BENCH, an interactive benchmark that replays enterprise telemetry through a live SIEM and requires each system to retrieve evidence. Across 1,247 alerts from a multi-stage attack scenario, every approach missed at least 40.4% of attack-related alerts. Trace analysis shows that attack alerts are more likely to be dismissed when searches return no records, same-context review has negative net correction, and dismissal receives no consistently stronger investigation than escalation. Based on these findings, we further design AIDA (Adversarial Investigation and Dialectical Analysis), a multi-agent framework that requires an explicit proposed decision before independent challenge and stronger evidentiary requirements before dismissal. AIDA preserves investigation history in an append-only Investigation Ledger and keeps the challenge in a separate reasoning context. A separate Judge adjudicates the proposed decision and challenge against evidence, resolving the alert or requesting another round when evidence is missing. On the same alerts, AIDA achieves an F1 score of 0.958, compared with 0.371-0.744 for the studied approaches, and reduces the false-negative rate from 40.4% to 3.1% while escalating 18.4% of alerts to analysts. These results show that structuring evidence retrieval and decision review can substantially improve agentic SOC triage.

0.45

RAVEN: A Self-RAG Based Agent for Threat Classification and Response

cs.CR ·2026-11-19 ·Pironti Francesco Aurelio, Arena Luigi, Blefari Francesco, Lupinacci Matteo, Romeo Francesco +1 more
similarity 0.61
Abstract

Security event analysis in large enterprises can be overwhelming, as each alert typically requires manual in- spection by human operators. This process is time-consuming and prone to inconsistencies, especially in high-volume Secu- rity Operations Center (SOC) environments. In this work- in-progress paper, we present RAVEN: Retrieval-Augmented Vulnerability & Event aNalysis, an agentic framework designed to support SOC analysts by automating the initial stages of alert investigation. The proposed system leverages a Self- Reflective Retrieval-Augmented Generation-based approach to provide an initial classification of security events and enables interactive refinement through analyst-in-the-loop queries. Un- like traditional approaches that rely on generic mitigation strategies, RAVEN generates context-aware remediation sug- gestions by incorporating knowledge of the target enterprise infrastructure. We implement an early-stage prototype and report preliminary results on three balanced test sets of 400 curated web payloads each (SQLi, XSS, SSTI) and a qualitative case study of mitigation generation; an evaluation on IDS- generated alerts and a user study are ongoing and outlined as remaining work.

0.12

INSIGHT-HPC: An Interpretable System Intelligence for Global HPC Telemetry

cs.DC cs.LG ·2026-11-02 ·Rupak Kadel, Jie Li, Yong Chen
published similarity 0.61
Abstract · Accepted at the 2026 IEEE International Conference on Big Data (IEEE BigData 2026)

Modern high-performance computing (HPC) clusters integrate multi-generation CPUs and GPUs, which generate rich, high-dimensional telemetry streams from out-of-band baseboard management controllers (BMC), job schedulers, and facility power distribution units. Detecting anomalies within this telemetry is critical to prevent energy waste and costly node failures. Existing approaches often rely primarily on statistical deviations and require stronger labels than actual production environments can provide. In addition, these detectors have a limited operational context for interpreting alerts. We present INSIGHT-HPC, a multi-plane anomaly-detection architecture that combines statistical, workload-aware, and physics-guided evidence into operator-interpretable episodes. INSIGHT-HPC integrates heterogeneous, large-scale telemetry streams and transforms noisy minute-resolution observations into de-duplicated fault episodes with operator-facing explanations. This design allows the system to distinguish workload-explained behavior from genuine infrastructure anomalies while producing interpretable alert summaries for operators. On production HPC telemetry with weak operational labels and controlled HPC-relevant fault injections, INSIGHT-HPC achieves a stronger precision-recall balance than all evaluated baselines. The system achieves a precision of 0.913, a recall of 0.817, an F1 score of 0.862, and an AUROC of 0.896. These results demonstrate the value of combining workload context, physical consistency, and explainable episode-level fusion.

0.37