CS Paper Radar

Know the Normal, Track the Attack: Context-Grounded and Stateful LLM Investigation over System Provenance

Lijie Zheng, Ji He, Ying Wang, Huang Zhang, Yulong Shen

cs.CR ·submitted 2026-09-29 01:49 UTC · v1

Abstract

Provenance-based intrusion detection systems (PIDSs) identify suspicious activity in audit streams, but their outputs remain difficult to turn into coherent attack narratives. Direct LLM analyses of local anomalous subgraphs lack deployment-specific normal-behavior knowledge and validated attack state across evidence fragments. This can cause unsupported attack interpretations of routine activities and incorrect attribution of temporally dispersed evidence to attack stages. We present ANCHOR, an investigation-oriented provenance system that combines evidence curation with context-grounded LLM reasoning. It calibrates anomaly judgments by relation type and links anomalous windows through rare relation-role patterns. The resulting evidence queues preserve causal structure, temporal boundaries, and cross-window continuity. The investigator interprets process-centered evidence using two complementary forms of context. Deployment Context combines environment-specific interaction and object baselines with high-risk security knowledge. Case Context uses a confidence-gated Attack-Tracking Cache to maintain investigation state across windows. Correlating current evidence with high-confidence prior findings, ANCHOR incrementally reconstructs attack narratives organized by kill-chain stages. We evaluate ANCHOR on six DARPA Transparent Computing E3/E5 datasets across three operating systems. Controlled evidence-level and end-to-end comparisons show improved overall IoC recovery and attack-stage attribution over state-of-the-art provenance-based baselines. These gains persist under a fixed LLM backbone in our evaluation. ANCHOR processes a full audit day at dollar-level API cost, supporting practical, context-grounded investigation across windows.

Comment
19 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Intrusion Detection for Agentic Processes: Evidence-Based Runtime Monitoring

cs.CR ·2026-09-10 ·Arslan Brömme
similarity 0.70
Abstract · 7 pages, 3 tables

Agent deployments increasingly combine language-model inference with retrieval, delegation, tool execution, external-system access, and human approval. Security-relevant deviations can therefore emerge across an evolving process rather than in one isolated input or action. Building on the author's earlier black-box architecture for agentic processes and the subsequent evidence-claim model, this paper proposes an Agentic-Process Intrusion Detection System (A-IDS), an evidence-aware security interpretation layer for runtime intrusion detection whose monitored object is the agentic process itself. A-IDS compares evidence-supported observations with a governed and versioned expectation baseline for workflow state, authorization, communication, and mandatory events. Its conceptual contribution combines dynamically due governed expectations, visibility separated from three-valued matching, explicit unresolved observation states, and bounded findings that separate evidentiary status from operational impact. The model further identifies the monitoring plane itself as an attack surface when adversarial content reaches semantic evidence producers through otherwise legitimate observation paths. Some observations may be produced outside the operational agent's self-report path, but the model does not assume complete observability or universally trustworthy capture. Prompt injection is treated both as an input-security problem and as a possible origin of later process deviations and cross-agent influence paths. A-IDS does not infer malicious intent from anomalous behavior, does not treat an unobserved event as proof of non-occurrence, and does not claim a new anomaly detector, temporal logic, or provenance model. The contribution is conceptual: it does not validate an implementation, demonstrate empirical detection performance, establish causal attribution, or provide an enforcement mechanism.

0.26

SecLog-Bench: A Scalable Benchmark for Cybersecurity Log Reasoning with Scaffolded Evidence

cs.CR cs.AI ·2026-10-22 ·Yifan Wang, Dongliang Xu, Xiaoyong Li, Jing Li, Peter S. Pan +1 more
benchmarkcode/data similarity 0.69
Abstract · 17 pages, 9 figures. Submitted to IEEE Transactions on Dependable and Secure Computing. Code and data: https://github.com/SecLog-Open/SecLog

Large language models (LLMs) have shown strong potential for cybersecurity, leading to growing interest in LLM-based automated cybersecurity log analysis. However, current datasets (1) are limited in scale; (2) rely mainly on single-source evidence; and (3) have only binary or multi-class attack labels, without dense annotations of the reasoning process. To address these limitations, SecLog-Bench has three key features. First, we build an extensible simulation platform that covers 29 attack scenarios, supports repeated execution and the integration of new scenarios, and enables scalable generation of cybersecurity logs with attack-stage labels. Second, we use multi-source logs, including audit logs, application logs, and network traffic, to jointly analyze cyber attacks. Third, we annotate each sample with the essential context for cybersecurity log reasoning and a structured step-by-step evidence chain. We evaluate eight mainstream LLMs on this benchmark for post-detection attack-progress reasoning. Results show that, when provided with scaffolded evidence, current LLMs exhibit modest capability in attack-progress reasoning, with the best model achieving only 44.44% accuracy and 45.07% Macro-F1. Their reasoning remains unreliable; i.e., the highest Stage Reasoning F1 (SRF1) score is only 54.87%. Moreover, even correct predictions may be accompanied by flawed reasoning, and reasoning quality is often misaligned with prediction accuracy.

0.35

TGL-APT: Temporal Graph Learning with Graph Distillation for Efficient APT Investigation

cs.CR ·2026-08-20 ·Jing Chen, Ayong Ye, Yuanhuang Liu, Yuexin Zhang
similarity 0.69
Abstract

Advanced Persistent Threat (APT) attacks pose a critical challenge to modern systems, as their stealthy, multi-stage nature renders conventional detection methods ineffective. While provenance graphs provide rich behavioral context for attack investigation, attack-relevant evidence is often sparse and embedded in large volumes of routine system activity, making full-graph learning both computationally expensive and difficult to correlate over long attack sequences. We present TGL-APT, an adaptive investigation framework built on the observation that attack-relevant information is non-uniformly distributed and often mediated by structurally influential or behaviorally distinctive entities, which we characterize as information-bottleneck nodes. TGL-APT combines three complementary components: (1) information-bottleneck-guided graph distillation that suppresses provenance redundancy while bounding structural distortion and preserving causal reachability; (2) adaptive temporal graph learning that continuously refines the core node set as node relevance evolves; and (3) cross-spatiotemporal attack fingerprint alignment that associates fragmented suspicious activities across different entities and time windows. Finally, causal expansion and stage characterization reconstruct coherent attack processes for investigation. Experiments on three DARPA E3 datasets show F1-scores of 95.7%, 90.9%, and 88.9%, while reducing training time, detection latency, and memory usage by approximately 39%, 33%, and 22%, respectively, compared with KAIROS. These results demonstrate that TGL-APT effectively balances detection performance, computational efficiency, and investigation capability for provenance-based APT analysis.

0.12

CyberClear: A Benchmark for LLM Agent Systems on APT Attack Chain Provenance

cs.CR cs.AI ·2026-09-26 ·Qi Chen, Fushuo Huo, Hangli Shen, Jingcai Guo, Shuhao Li +1 more
benchmark similarity 0.69
Abstract

Large language model agents have demonstrated promising capabilities in cybersecurity tasks, yet their ability to reconstruct complete Advanced Persistent Threat attack campaigns from complex security logs remains largely unexplored. Existing cybersecurity benchmarks for agents mainly focus on vulnerability discovery, exploitation, and security analysis tasks, leaving the evaluation of attack chain provenance under realistic security logs insufficiently studied. To address this gap, we introduce CyberClear, a benchmark for evaluating LLM agents and advanced agent systems on APT attack chain provenance from long-context security logs. CyberClear covers both single-step attacks and multi-stage attack chains, requiring agents to identify attack evidence, infer attack progression, and generate provenance graphs containing entities, causal relationships, MITRE ATT&CK techniques, and forensic evidence. To enable comprehensive evaluation, we develop an evaluation method tailored to APT attack chain provenance. Unlike conventional text similarity metrics that focus on surface-level matching, our evaluation examines whether reconstructed graphs preserve the semantics of attack chains across single-step behavior correctness, multi-step behavior identification, temporal and causal consistency, entity and relationship fidelity, and overall attack narrative consistency. Advanced multi-agent systems powered by state-of-the-art LLMs still struggle on CyberClear, motivating us to propose CyberProvenance, an agent cyber harness designed for multi-agents that augments LLM agents with evidence accumulation, execution-based validation, and feedback-guided refinement mechanisms for reliable attack-chain provenance. Extensive evaluations on CyberClear demonstrate the effectiveness of CyberProvenance in improving evidence reasoning, execution-grounded validation, and complete APT attack chain reconstruction.

0.07

SoK: Timestamps Are Not a Time Model: Rethinking Temporal Reasoning in Host-Based Intrusion Detection

cs.CR cs.AIcs.LO ·2026-10-26 ·Rene Heinzl, Markus Nissl
survey similarity 0.68
Abstract · Extended version with full appendices (search method, KRR domain analysis, rating evidence, formalizations). 51 pages, 1 figure, 16 tables. …

Host-based intrusion detectors order events, window audit streams and sequence attack stages by their timestamps. Every conclusion they draw therefore rests on assumptions about clocks, delivery, entity identity and the integrity of the evidence, and most of these assumptions go unstated. This Systematization of Knowledge (SoK) makes them explicit. Drawing on temporal databases, stream processing and runtime verification, we distill five components of a time model, together with a checklist of evidence assumptions and four levels of guarantee, and walk through a running campaign with them. A documented search yields fifteen families of detection and investigation. For papers selected by screening role and period, plus named systems and landmarks, we check the stated assumptions in the full texts, and we assess eleven temporal knowledge representation and reasoning (KRR) formalisms and a learned baseline on nine attack patterns. Some early alert correlators stated clock and delivery assumptions, and forensics measures clock error. Current provenance-based detectors state no clock bound. Four audited papers describe input-ordering conditions or conventions, but none specifies conditions for conclusion validity across a stated class of delivery failures. In the selected provenance-based systems, campaign composition uses system-specific procedures or objectives, learned state, causal graph queries or language models, and beyond CEP-based query systems, we found no temporal rule language or stream reasoner from KRR applied to host audit data. In 28 of 38 audited provenance-era detector papers, capture or stored evidence is trusted or its manipulation is placed out of scope, and none says which conclusions survive deleted, delayed or forged records. Five open challenges follow from these gaps.

0.21

Back to Basics: Formal Analysis Hints Improve Weak-to-Strong Monitoring

cs.CR ·2026-10-25 ·Daniel Ben-Levi, Judah Goldfeder, Mateusz Dziemian, Raja Moreno
similarity 0.67
Abstract

Frontier agentic systems are capable of taking misaligned actions that may cause significant harm. Monitoring techniques are developed to oversee these systems in order to detect and prevent dangerous actions. However, current state-of-the-art monitors suffer from four persistent failure modes: complete detection failures, partial detections, accepting false benign framings, and suspicion score calibration errors. These failure cases can be efficiently combated by augmenting transcripts with supplementary hints, spanning command provenance, security-sensitive action flags, and task-disallowed effect markers. Provenance links allow for the recognition of diffuse attacks, while security and authorization flags simultaneously enable subtle attack recognition and provide an explicit reason for monitors to doubt the false benign framings they encounter, reducing rationalization and improving calibration. We find that augmenting weak extract-and-evaluate monitors with these hints significantly boosts performance with minimal computational cost. Furthermore, a failure mode analysis confirms that hints can remediate all four failure modes, notably almost entirely eliminating detection failures. A series of ablation studies emphasizes the efficiency and potential of the augmented extract-and-evaluate approach. Synthesizing our findings, we propose a tractable two-pronged approach toward enabling weak-to-strong monitoring.

0.49

When AI Investigators Cannot Trust Their Tools: Conflict-Aware Evidence Verification for Cyber Threat Investigation

cs.CR cs.LG ·2026-10-30 ·Yuchen Zhang
published similarity 0.65
Abstract · Accepted at WAITI 2026 (3rd Workshop on AI for Cyber Threat Intelligence), co-located with ACSAC 2026

LLM agents increasingly triage security alerts by combining the outputs of several detection tools, implicitly treating those tools as trustworthy. Unlike prior work on conflicts between a model's knowledge and its context, we study conflicts among the tools themselves: what happens when this machine-generated evidence is conflicting or wrong. On the corrected CIC-IDS2017 dataset, four heterogeneous detectors emit labels and confidences that we corrupt under a threat model varying the number of corrupted tools, the persistence of the corruption, and the confidence attached to wrong outputs. Across open-weight LLMs from 3B to 14B parameters, unverified investigators are fragile: one corrupted detector out of four lowers naive accuracy to .50-.72, three lower every unverified investigator to .13 or less, and honest disagreement alone makes them abstain on up to 63% of flows. We propose Conflict-Aware Evidence Verification (CAV), built from established components: an equal-weight vote that ignores reported confidences, an evidence-conflict gate, and an independent nearest-neighbour trust check that tests the leading candidates. With its prompt frozen before testing, CAV-LLM improves on the naive investigator in every corrupted condition ($p < 0.001$), reaching .84-.88 with three corrupted detectors against at most .08, while calling the LLM on 0.4% of clean traffic. A fixed verification rule is as accurate or better unless most tools are corrupted. When we make the independent channel itself noisy, CAV inherits its errors: it stays ahead when two or more tools are corrupted, but adding the channel as an extra voter is better when at most one is. Verification thus moves trust from the tools to the check, which must be measured and protected in turn.

0.20

BEACON: Behavior-Anchored Cross-Source Knowledge Graph Construction for Cyber Threat Intelligence

cs.CR cs.CL ·2026-08-28 ·Changze Li, Yutong Cheng, Tsania Camila Finnisa, Qian Cui, Wei Ding +1 more
benchmark similarity 0.65
Abstract

Cyber threat intelligence (CTI) is foundational to modern cyber defense, yet much of it resides in unstructured reports whose volume and heterogeneity far exceed manual analysis, motivating research on automatically constructing knowledge graphs from CTI reports. However, existing approaches mainly extract partial information within a single report, leaving the cross-source setting unexplored, where the same threat is given unrelated names. Our key insight is that attack behaviors, once mapped to MITRE ATT&CK (a standardized catalog of attack techniques), can anchor the rest of a report. Attack behaviors are the adversarial actions a report describes, while contextual entities (e.g., threat actors, campaigns, and affected products) and Indicators of Compromise (IoCs; e.g., IP addresses) are their participants and traces. Attaching them to these anchors places every per-report graph in one canonical space. We realize this insight in BEACON, an LLM-driven framework for cross-source CTI knowledge graph construction. Its first stage extracts each report into a graph under a propose-then-verify paradigm, grounding candidates in report evidence and official ATT&CK definitions, to suppress LLM misclassification and hallucination. Its second stage merges these graphs with a hierarchical alignment strategy that applies signals in decreasing order of determinism, from character-level and semantic similarity to overlapping technique neighborhoods, iterating as merges pool neighborhoods. No existing benchmark links entities to technique anchors or provides cross-source alignment ground truth. We therefore construct and release two human-annotated datasets from 34 sources: to our knowledge the largest for report-level CTI extraction (8,395 elements) and the first for cross-source consolidation (3,487). On them, BEACON outperforms all baselines by at least 23% and 9%, respectively.

0.20