CS Paper Radar

Measuring LLM Agent Defenses by Layer: GDA Measurement and an Envelope-Layer Attack-Defense Model

Yuelin Han

cs.CR ·submitted 2026-09-30 11:19 UTC · v2

Abstract

Agent interaction protocols such as ACP and A2A let a remote peer deliver a task the receiving agent treats as legitimate, opening a channel for indirect prompt injection. Evaluating and defending such agents is difficult because the attack success rate (ASR) cannot distinguish an LLM recognition failure from an agent-layer block, and so cannot indicate which layer to strengthen. We contribute three artifacts. (1) GDA Measurement, a red-team method that captures the raw agent-LLM context at the gateway, preserves conversation and raw-capture data, and judges outcomes autonomously with an agent selected by an inter-annotator-agreement (IAA) study over six candidates; its four classes, A/B/C/D, replace ASR with the semantic refusal, semantic breach, interception and penetration rates - a layer-resolved view of defense. (2) A2A-TIBA, an implant-command-exfiltration principle on A2A that generates this method's validation corpus, designed to spread outcomes across the four classes rather than to maximize the success rate. (3) ELA-ITL, a three-layer isomorphic attack-defense model induced from that corpus, which identifies the envelope layer - the channel through which content enters an agent - as a defense surface; ablating the A2A, tool and memory channels shows that annotating untrusted content markedly improves recognition of malicious content.

Closest papers

Nearest neighbours by meaning across the whole index.

The Envelope Layer: An Underestimated Surface of LLM Agent Defense

cs.CR ·2026-10-27 ·Yuelin Han
similarity 0.73
Abstract

Prior work framed the envelope layer as the channel through which malicious content enters an agent, but did not examine the effects of label strength, envelope authenticity, scaffolds, or existing memories. Following its GDA Measurement and A/B/C/D adjudication, this paper takes the A2A-TIBA attack as the test vehicle and the semantic breach rate as the primary indicator, and characterizes the security boundary of the envelope layer through a 2x2 factorial experiment, a memory experiment, and control experiments (an unlabeled same-protocol baseline, a one-day interleaved control, and benign tasks). The results show that, under authentic envelopes, strong labels consistently reduce the semantic breach rate (-56 to -61 percentage points), and the benefit of weak labels is about a quarter to a half of that of strong labels; forging the envelope weakens the strong label's benefit (interaction OR = 2.10, p = 0.022) and its direction varies across frontends; the effect of memory valence exceeds that of any label manipulation; success memory raises the semantic breach rate and failure memory lowers it, while most conflict combinations return to the no-memory baseline; the interleaved control reproduces the direction that "unlabeled is worst"; and on benign tasks the cost of strong labels appears in layers across frontends and backends. Accordingly, this paper extends the envelope layer into "admission surface plus context assembly surface," whose security boundary is a dynamic outcome jointly shaped by label claims, frontend permissions, and historical memory.

0.11

Silent Failures in Agentic Security Evaluation: A Validated Harness for Tool-Call Mediation Under Indirect Prompt Injection

cs.CR cs.LG ·2026-09-26 ·Animesh Shaw
benchmarkposition similarity 0.72
Abstract · 7 pages, 4 figures, 3 tables

LLM agents that invoke privileged tools are vulnerable to indirect prompt injection (IPI), in which adversarial instructions embedded in retrieved data hijack the agent's actions. A growing body of work evaluates defenses against IPI, but the validity of that evaluation is rarely examined. We audit an IPI benchmark and its harness and identify four defect classes -- silent payload non-delivery, attack success scored by tool identity rather than arguments, false-rejection rate conflated with model incapacity, and the absence of an audit trail -- each of which yields a plausible, publishable, and incorrect number. We quantify the distortion by re-scoring identical execution traces under the defective and corrected definitions: on real agent behaviour, the tool-identity scorer reports a 21.7% attack-success rate where the true argument-level rate is 1.2%. In the sharpest case, an open model previously reported at 62.8% registers 0% under the corrected harness -- the prior figure largely an artifact of undelivered payloads and identity-level scoring. We release a harness whose construction makes each defect unrepresentable -- machine-checkable payload placement, argument-level attacker predicates, per-scenario environments, and mandatory trace persistence -- and use it to report three quantities the field does not: whether a compromised agent discloses the attack, the full security/utility operating curve of an LLM-judge defense, and tool-calling capability disentangled from defensive over-blocking. A corrected harness further overturns a reported "capability barrier": a model deemed incapable of tool use is in fact fully capable, its earlier result an artifact of environment mismatch. We argue that evaluation validity is a prerequisite for, not a footnote to, defense claims in agentic security, and provide an instrument that enforces it.

0.44

Weighing the Envelope: Component Weight and Arbitration in Envelope-Layer Defenses for a Single LLM Agent

cs.CR ·2026-11-18 ·Yuelin Han
similarity 0.70
Abstract

The envelope layer is the channel through which malicious content enters an LLM agent. Prior work established label and memory effects on multi-agent A2A envelopes, but the weight structure when MCP tools and memory coexist in a single agent remains unclear. Our threat model assumes an attacker pre-plants a malicious tool and poisons long-term memory; a benign user request inadvertently triggers it, and the attacker never enters the conversation. Following GDA Measurement and A/B/C/D, we measure how labels, memory, tool roles, and the backend shape invocation decisions across label-memory conflict, tool roles, robustness, and tool generalization. A tool-side forced-verification label substantially reduces semantic breach, but strong success memory pulls it back; a memory-side downgrade statement returns the outcome to the level of the label alone. The label works through mandatory implementation inspection plus default denial, not length or format; memory authorization wording erodes it with dose, while position and multi-turn pressure do not change the defense direction. Tool roles divide label effects into substitution, suppression, and boundary failure: low penetration is not safety, local leakage is the main residual risk, and a second malicious tool shows the same direction. Envelope labels are not a switch but a measurable, reversible weight constrained by backend and tool role, acting as a soft gate and risk signal within a broader defense rather than a substitute for permissions, sandboxing, or human review.

0.10

How Do LLM Agents Actually Get the Flag? Trace-Level Provenance for Agentic Offensive Security Evaluation

cs.CR cs.AI ·2026-08-26 ·Kimberly Milner, Minghao Shao, Nanda Rani, Haoran Xi, Venkata Sai Charan Putrevu +6 more
code/data similarity 0.69
Abstract

Capture-the-Flag (CTF) benchmarks are widely used to assess the offensive security capabilities of autonomous language-model agents. Evaluations rely on shallow binary judgments or aggregate scores, overlooking the agent's trajectory to the flag. Consequently actual exploitation is conflated with direct flag exposure, memorized recall, external lookup, guessing, and unsupported claims, potentially overstating the agent's cybersecurity capability. We introduce CTF-ABACUS, a trace-based agent auditing framework that reconstructs each run as an evidence-grounded solve profile. By decomposing agent actions into penetration-testing phases and categorical techniques, it identifies where exploitation occurs, where the flag first appears, and whether the recovered flag is supported by demonstrated behavior. Aggregating solve profiles across agents yields challenge signatures that reveal whether success was achieved via the intended exploit or via shortcut pathways. We apply CTF-ABACUS to 1,435 CTF attempts by six frontier and open-source models on 240 challenges, yielding 2,870 solve profiles under two judge lenses. Trace-verified exploits account for only 62-87% of recovered flags across benchmarks, while shortcut recoveries follow substantially shallower trajectories. These findings shift CTF evaluation from counting recovered flags to verifying demonstrated exploitation and provide a basis for designing benchmarks that better isolate the offensive capabilities.

0.49

Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security

cs.AI cs.CLcs.CRcs.MA ·2026-10-06 ·Shaswata Mitra, Raj Patel, Subash Neupane, Sudip Mittal, Md Rayhanur Rahman +1 more
similarity 0.69
Abstract · 26 pages, 20 figures, 24 tables

LLM-based multi-agent systems (MAS) engage tools, share memory, and delegate tasks, often encountering adversarial content. Current defenses for MAS are typically evaluated in isolation, focusing on one attack type at a time, which can lead to costly and hard-to-audit outcomes. This study organizes defenses into five principles, implementing them as DEFER1 (DEterministic-First Enforcement with Residual judgment), which includes a cascade of 28 checks that blocks what it can and refers the rest to a panel of four judges. In independent testing across four domains, attack success rates drop from about 30.0% to approximately 3.0%, with 78% of blocked attacks handled by deterministic checks. Only a quarter of proposals reach the judges in the security-operations domain, illustrating that the rules provide security for attacks violating clear policies, while judges manage those that only misrepresent intent. Both systems have weaknesses, such as a risk-score approval gate that inaccurately approves most attack proposals but few legitimate ones, highlighting the challenges in assessing threats accurately.

0.14

ORBIT: A Framework for Multi-Agent Safety and Security Evaluations

cs.MA cs.CR ·2026-09-27 ·Ben Hagag, William L. Anderson, Srija Chakraborty, Christian Schroeder de Witt
benchmarkcode/data similarity 0.68
Abstract

Multi-agent LLM systems are increasingly deployed for complex, long-horizon tasks or emerge as a natural consequence of agents interacting in the wild. Yet they give rise to significant safety and security risks: the flexible protocols that enable task generalization also expose novel threats, from cascading prompt injection to inter-agent collusion. Progress in defending against these threats has been slowed by a lack of shared empirical infrastructure, which forces bespoke environment development for every new defense and makes standardized comparison impossible. Existing evaluations address isolated threat models or single-agent settings, but none jointly vary attack, defense, and architecture across realistic multi-agent environments. To address this gap, we introduce ORBIT, a configurable evaluation framework for empirical multi-agent safety and security research, built on UK AISI's Inspect. ORBIT lets researchers configure communication topologies, memory, scheduling, and agent roles. It supports four threat types and four defense strategies, as well as non-adversarial failures, with a benchmark suite spanning five scenario families covering browser use, computer use, agentic coding, customer service, and cooperative allocation. Our central finding is a gap in defense transferability across threats: per-action defenses that cut a compromised agent's attack success by 60 points on multi-issue coding give no measurable protection against colluding agents, and none of the defenses we tested generalized over all attacks tested. We further demonstrate security-performance tradeoffs and interactions between architecture and defense effectiveness. We make ORBIT available open-source at https://github.com/wlanderson0/orbit.

0.46

We Have All the Cards: Securing Agent Cards at the Operational Trust Boundary of the Agent2Agent Protocol

cs.CR cs.AI ·2026-11-13 ·Miguel A. Guirao Aguilera, Satyajayant Misra, Efren Lopez-Morales
publishedbenchmark similarity 0.68
Abstract · To be published at the Workshop on Agentic AI in Offensive and Defensive Cyber Operations (AIDC'26) proceedings co-located with the IEEE Ann…

AI agent interoperability protocols enable orchestrators to discover and invoke external agents using machine-readable metadata. In the Agent2Agent (A2A) protocol, Agent Cards contain attacker-controllable natural-language fields that may influence agent selection before a provider is invoked, creating a pre-invocation trust problem. We characterize this discovery-time attack surface through a taxonomy of ten Agent Card manipulation attacks and present SECCARD, a layered validation framework combining protocol checks, deterministic analysis, and LLM-based semantic reasoning. We measure the emerging public A2A ecosystem by probing 36.1 million Internet domains and collecting 310 deployed Agent Cards. Two annotators independently label the corpus with 96.8% agreement. We then construct a 660-card real-card perturbation benchmark with matched benign controls and a 260-card synthetic adversarial benchmark spanning all ten attack categories. On controlled attacks injected into real Agent Cards, the strongest semantic judge achieves 97.0% security recall at a 1.7% false-positive rate. Across nine LLM judges on the synthetic benchmark, security recall ranges from 25.8% to 92.3%, demonstrating substantial model-dependent variation. These results show that SECCARD effectively identifies Agent Card manipulations spanning all ten discovery-time attack classes while preserving benign Agent Cards, but also reveal that its effectiveness depends critically on the choice of the underlying LLM judge.

0.32

Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluation

cs.CR ·2026-09-21 ·Chetan Pathade, Prathamesh Pawar, Shubham Patil
benchmarkposition similarity 0.67
Abstract · 7 pages, 2 tables, 2 Figures

Attack success rate (ASR) is the headline metric in nearly every published evaluation of attacks on, and defenses for, LLM agents. We argue that ASR as currently used is not a single quantity but a family of metrics parameterized by six design choices that papers seldom specify and never hold constant across the literature. We support this with two studies that require no proprietary access. First, a full-text meta-analysis of 259 agentic-security papers posted to arXiv between February 2025 and September 2026 finds that most report neither a variance estimate nor repeated runs for their headline attack metric: 58% (95% CI 44-71) in a hand-coded random sample of 50, 65.3% by automated coding of all 259. Only 30.9% disclose enough about decoding to establish whether their evaluation was even stochastic, and of the 64 papers we confirm use an LLM judge, 29.7% report any agreement check against human labels. Second, an analytical study shows that these omissions are not cosmetic: on a 100-instance benchmark, the minimum difference in ASR detectable at conventional power is 18.2 percentage points, and two defenses whose true ASRs differ by 5 points are ranked in the wrong order by a single-run evaluation roughly 21% of the time. Because several of the six axes shift ASR in a system-dependent way, the resulting incomparability is not a constant offset that cancels in comparison. We conclude that cross-paper ASR comparison is currently unsupported, and propose a ten-item reporting checklist targeted at each failure we measure. Our aim is not to dispute any individual result but to supply the shared measurement contract the field has so far done without.

0.52