CS Paper Radar

Evaluating autonomous self-healing against persistent cyber-attacks

Bart Gijsen, Stefano Nieuwenhuizen, Lucien Yap, Georgi Spasov, Erwin de Haan, Mahira Ali, Shari Finner

cs.CR ·submitted 2026-11-02 10:00 UTC · v1

Abstract

The need for autonomous cyber resilience (ACR) is strongly accelerating. The boost in cyber capabilities of advanced AI is speeding up the attacker-defender rat race to an unprecedented pace. As a result, the field of ACR must progress from early design and experimentation to hands-on evaluation and deployment. To illustrate what is needed for this transition, we put TNO's autonomous immune system-inspired Self-Healing for Cyber Security (SH4CS) software to the test. Our analytic and experimental evaluation results demonstrate that effective cyber resilience is achieved with SH4CS for (repeated) web application vulnerability exploits. Remaining challenges of SH4CS revolve around finding the right balance between predictable resilient behaviour, and adaptiveness to the large variety of cyber threats and applications-to-be-protected.

Comment
Presented at AutonomousCyber 2026 workshop on 18 September 2026 and accepted for the workshop proceedings

Closest papers

Nearest neighbours by meaning across the whole index.

Self-Healing by design: Immune-system inspired autonomous cyber resilience

cs.CR ·2026-11-08 ·Shari Finner, Swarna Kumarswamy-Das, Bart Gijsen
published similarity 0.75
Abstract · The content of this paper was presented at MAST Australia 2024, 19-21 November in Adelaide. The presentation was accepted after review of th…

Digital innovations have strongly improved ship capabilities, including command, control & communication, navigation and combat management. In the wake of digitisation and geopolitical stress, resilience against cyber-attacks has become a key challenge. Autonomous Cyber Resilience (ACR) methods, inspired by features of the human immune system, provide a promising perspective. Proofs of concept of autonomous self-healing and self-protecting software have been realised in cloud environments using containerisation, monitoring and deployment technology. However, developing a suitable "cyber immune system" for more complex digital systems is still an intricate challenge. In this paper, we share insights on how ACR methods can be combined with resilience mechanisms present in critical naval and aeronautical manoeuvring systems. Using TNO's "OpPic" application, we show how TNO's Self-Healing for Cyber Security software can provide autonomous mission- and threat-adaptive resilience. On this basis, six guidelines for the design of autonomous cyber resilient systems are derived. The ultimate goal is to enable operators to oversee and control their system's "cyber immune response", to be able to continuously adapt to evolving threats and mission priorities.

0.63

CyberPersistBench: Evaluating LLM-Based Cyber Attackers on Installation and Persistence

cs.CR ·2026-09-29 ·Sujin Chen, Lijun Li, Xuhong Wang, Jing Shao
benchmark similarity 0.63
Abstract · 29 pages, 15 figures

While LLM-based attackers exhibit growing proficiency in vulnerability exploitation, most existing cybersecurity benchmarks suffer from single-stage truncation, prematurely terminating evaluation upon initial access. In practice, initial footholds are exceptionally fragile across operational disruptions such as service restarts and host reboots. Whether LLM-based attackers can establish and maintain durable footholds beyond initial compromise remains a central blind spot in cybersecurity evaluation. To bridge this gap, we introduce CyberPersistBench, the first benchmark dedicated to post-compromise installation and persistence. Decoupled from upfront exploitation, CyberPersistBench frames persistence as an adversarial survival task in which agents use native host mechanisms to maintain footholds across staged system disruptions. Deterministic checks support a six-level scoring method (L1--L6) spanning installation and persistence. The benchmark comprises 203 core tasks across seven categories, augmented by multi-host and active defense extensions. Empirical evaluations across five frontier agents show that autonomous persistence remains limited (27.6%--44.8%) and drops further on defense-enabled tasks (5.5%--13.3%); nonetheless, these results reveal an emerging cyberattack risk, underscoring the necessity of benchmarking post-compromise persistence. CyberPersistBench thus establishes a foundational benchmark for post-compromise installation and persistence, delineating the operational boundaries of autonomous cyber agents.

0.21

ReproBench: Benchmarking LLM Agents on Reproducing Vulnerability From Scratch

cs.CR ·2026-09-28 ·Liang He, Sheng Wu, Haomiao Hao, Hongduo Zhao, Jia Yan +1 more
benchmark similarity 0.62
Abstract · 15 pages (9 pages main text + 6 pages appendix), 5 figures, 11 tables. Submitted to AAAI 2027

Large language model (LLM) agents are increasingly evaluated on cybersecurity tasks such as vulnerability reproduction, exploitation, and patching. However, existing cybersecurity benchmarks predominantly operate under a post-environment evaluation paradigm, i.e., handing the agent source code, a container, or an executable binary. This setup bypasses the critical environment reconstruction step, leaving a fundamental question for real-world vulnerability analysis: can an agent autonomously reconstruct the required execution environment and reproduce a vulnerability entirely from scratch? To address this gap, we present ReproBench, an evidence-grounded benchmark designed to evaluate agent capabilities in end-to-end vulnerability reproduction starting from solely a CVE identifier. ReproBench decomposes the full reproduction workflow into six distinct phases, and assesses performance on each phase independently using verifiable experimental artifacts: downloaded firmware images, unpacked binaries, granular analysis logs, and validated crash samples, among others. We instantiate ReproBench with 30 real-world IoT firmware vulnerabilities, which serve as ideal test cases for our from-scratch evaluation setting. Our evaluation demonstrates that 45.3% of test runs resort to vulnerability simulation - a prevalent remediation workaround adopted across all evaluated LLM agents - while only 5.3% of CVE-model pairs achieve successful reproduction of real-world vulnerabilities. Despite the low overall success rate, these non-trivial successful cases confirm that state-of-the-art LLM agents already possess the capacity for fully autonomous end-to-end vulnerability reproduction. Concurrently, our in-depth analysis of failed cases identifies core bottlenecks impeding LLM agents throughout the reproduction pipeline, offering actionable insights for subsequent research.

0.41

Agentic AI Cybersecurity Framework

cs.CR ·2026-08-20 ·Victor Kebande
similarity 0.62
Abstract

The increasing scale, complexity, and dynamism of modern cyber threats have rendered traditional reactive cybersecurity mechanisms insufficient. This paper introduces an Agentic AI Cybersecurity Framework (AACF) designed to enable autonomous, goal-driven, and adaptive cyber defense operations. Unlike conventional systems that rely on predefined rules and human intervention, the proposed framework leverages agentic artificial intelligence to perceive environmental states, reason about potential threats, and execute context-aware responses with minimal supervision. The framework is structured into key functional layers, including perception, reasoning, decisionmaking, action execution, and feedback-driven learning, enabling continuous adaptation to evolving attack patterns. By integrating intelligent agents with real-time data analysis and automated response mechanisms, AACF supports proactive threat detection, dynamic risk assessment, and coordinated mitigation strategies across distributed environments. A conceptual architecture is presented, along with illustrative use cases demonstrating its applicability to intrusion detection, incident response, and autonomous security orchestration. The proposed framework contributes to the emerging paradigm of self-directed cybersecurity systems and provides a foundation for developing resilient, scalable, and intelligent defense infrastructures.

0.19

Beyond Handcrafted Security: Towards Self-Evolving Defense for LLM Agents

cs.CR cs.AI ·2026-08-13 ·Jiajun Ruan, Peiyang Li, Yukun Chen, Fengting Li, Chao Feng
similarity 0.62
Abstract

The expanding operational capabilities of large language model (LLM) agents introduce sophisticated security threats. Runtime defenses have emerged as an effective approach to mitigating these risks by integrating security mechanisms into the agent execution loop. However, existing runtime defenses rely heavily on manually designed interventions and lack a principled framework for their construction and maintenance. In this work, we first develop a harness-level formulation of runtime defense that systematically characterizes how harness mechanisms enable defense construction and provides a unified view of existing runtime defense interventions from a harness perspective. Building on this formulation, we propose HARD (Harness-based Autonomous Runtime Defense Evolution), a self-evolving runtime defense framework that automatically identifies appropriate intervention strategies and iteratively improves defense artifacts based on observed failure traces. HARD transforms runtime defense development from manual engineering into an autonomous evolution process, and extensive experiments demonstrate that it improves security performance over existing handcrafted defenses while preserving benign task utility. Our findings highlight autonomous defense evolution as a promising new paradigm for securing deployed LLM agents, enabling agents to identify defense weaknesses and continuously improve their protection mechanisms.

0.41

CyberBinGym: Can AI Agents Turn Binary Patches into Real Exploits?

cs.CR cs.AIcs.LG ·2026-11-07 ·Lambang Akbar Wijayadi, Zhaoqi Xiao, Zhun Wang, Hongwei Li, Wenbo Guo +3 more
benchmark similarity 0.61
Abstract

Security fixes usually reach users as compiled updates, but deployment delays leave unpatched systems exposed. During this window, comparing patched and unpatched binaries can reveal how to exploit a vulnerability, turning a defensive update into actionable information for attackers. Existing benchmarks do not measure this capability: they expose source code, rely on human-crafted challenges, or evaluate binary comprehension without requiring exploitation. We introduce CyberBinGym, a benchmark containing 100 tasks derived from real vulnerabilities in 53 widely used projects, spanning six vulnerability families and three levels of analysis complexity. Agents receive stripped vulnerable and patched binaries without source code or vulnerability information, and must combine static and dynamic analysis to recover and exploit the vulnerability. Evaluation separately measures vulnerability recovery through differential crash testing and end-to-end exploitation through flag retrieval, with trajectory audits to reject reward hacking. The strongest evaluated agent recovers the target vulnerability in 88% of tasks and exploits it in 20%. Providing the patched binary raises its exploit rate from 10% to 25% on a matched 20-task subset; with standard binary hardening enabled, it still exploits two of its 20 previously successful tasks. These results demonstrate that current agents can turn released binary patches into working exploits in a controlled setting, although exploitation remains unreliable. These capabilities lower the barriers to targeting unpatched software and compress defenders' effective response window, increasing the urgency of timely patch deployment.

0.13

Does AI Help Cyber Attackers or Defenders? Evidence from Nonpublic Vulnerabilities and Subsequent Attacks

cs.CR cs.AI ·2026-10-05 ·Tobias Heldt, Matt Turk, Christoph Landolt, Mario Fritz
similarity 0.61
Abstract · 8 pages, 1 figure

The release decision for frontier AI systems increasingly relies on cyber capability benchmarks, yet public vulnerability benchmarks can expose agents to previously published advisories, exploits, and fixes, making it difficult to distinguish prior exposure from capability on unseen vulnerabilities. We evaluate open-weight and proprietary AI models on exploit generation, vulnerability repair and subsequent attacks in five nonpublic software environments, including vulnerabilities we privately disclosed while they remained unpatched. Researcher-developed and reviewed deterministic graders, not LLM judges, determine task scores. Comparisons with 209 disclosed vulnerabilities and cryptographic challenges reveal substantial variation across systems and vulnerability types. Repair scores exceed attack scores in two nonpublic environments and fall below them in three. Passing an initial security test is also insufficient: another exploit succeeds in 92 of 524 non-independent defender test intervals after the initial exploit is stopped. These results motivate vulnerability-specific attack-repair comparisons and subsequent resistance tests.

0.26

Self-Evolving Defense: Continual Security Policy Learning for LLM Agents

cs.CR ·2026-09-29 ·Minh Nhat Le, Nisarga Gondi, Yibo Peng, Ronghao Ni, Limin Jia +2 more
code/data similarity 0.61
Abstract · 39 pages, 6 figures. Code: https://github.com/Infini-AI-Lab/SED. Content warning: this paper contains examples of harmful and offensive lang…

Large language models (LLMs) increasingly power agents that access sensitive information, use external tools, and modify software repositories. Although these capabilities offer substantial benefits, they also create security risks such as jailbreaks, prompt injection, and vulnerable code generation. Existing defenses often require retraining, fail to adapt to evolving attacks, or address only a single threat pattern. To address these limitations, we propose Self-Evolving Defense (SED), a training-free framework that distills harmful agent trajectories into reusable security policies without updating model weights. By retrieving relevant policies for future tasks, SED continually adapts to new attacks while retaining knowledge across attack scenarios. To evaluate the effectiveness of SED, we test it with three open-source models (DeepSeek V4 Flash, GLM 5.2, and Kimi K3) on eight benchmarks that span jailbreaks, prompt injection, and insecure code generation. SED lowers targeted prompt-injection success on AGENTDOJO to 0.42%, compared with 3.7% for the best baseline defense, and holds adaptive X-TEAMING attack success on HARMBENCH to 7.8%, more than four times lower than the best baseline at 35.2%, while preserving benign task utility.

0.34