CS Paper Radar

What Victims Take Away: Attack Reuse and Defense Revision after Prompt-Injection Attacks

Zihan Chen, Lei Nico Zheng, Di Zhu

cs.CR cs.HC ·submitted 2026-10-27 00:04 UTC · v1

Abstract

Attack logs and red-team reports show targets the attacks used against them, to help them repair their defenses. Prompt injections against large language models (LLMs) are plain text, so such records also hand targets a working copy. Drawing on social learning theory and work on the victim-offender overlap, we study what targets do with that copy in Tensor Trust, an online game whose players defended LLM-guarded accounts, attacked one another's, and could read every attack they received. Using all 563,349 attack records, we compare each received attack with matched attacks that circulated among similar players at the same moment but never reached the target. Reuse mainly followed success: within a week, targets attacked other players with 10.0% of the attacks that had broken their defense, versus 0.03% of matched attacks (adjusted difference: 8.8-10.0 percentage points). Near-misses, whose replies began with "access granted" but that scored as failures, were reused about as rarely as failures: reuse tracked whether an attack paid off, not how its reply read, consistent with payoff-biased social learning. Breaches were also followed by defense changes, after which same-family attacks became 24.5 percentage points less successful against the target, relative to other defenders. Reuse came mainly from players already attacking, not newcomers. Previously received texts accounted for 23.2% of the successful hijacks we analyze. The attack history thus served both repair and reuse; we derive testable hypotheses for how AI security tools should disclose attacks, such as describing the technique without the full text.

Comment
20 pages, 5 figures, 10 tables. Working paper

Closest papers

Nearest neighbours by meaning across the whole index.

LLM-Attack Suite: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM Systems

cs.CR cs.DCcs.LG ·2026-10-15 ·Georgios Politis, Evangelos Pappas
similarity 0.59
Abstract

Split language models keep input text on a client but send intermediate representations to a server. Attacks can reconstruct text from these messages. Comparing privacy evaluations is difficult when attackers see different messages, know different model weights or use different datasets and recovery measures. We present LLM-Attack Suite, a framework that records these choices and lets researchers check, compare and extend an evaluation. Compatible attacks reuse recorded messages, and their predictions are saved for independent scoring. The scoring rule requires a linked undefended reference before a defence can pass. Reports also identify attacks that remain untested. In a controlled GPT-2 experiment, adding gradients returned during training increased token recovery from 94.20% to 97.38% and exact-document recovery from 13.71% to 37.77%. The 95% interval for the token gain lay above the two-percentage-point threshold fixed before testing. This experiment is shared with a companion study; here it tests how the evaluation responds to a change in attacker access. Further studies measure how corpus choice and compression defences affect recovery, compare an adapted published attack with its reference implementation, and test the measurement code. The framework reports implementation checks separately from measured attack effectiveness. Each privacy claim should state the attacker, dataset and scoring rule, report its controls and identify untested attacks.

0.55

SoK: PIAtlas: Decomposing the Unbounded Prompt Injection Attack Space

cs.CR cs.CLcs.LG ·2026-10-18 ·Stavros Damianakis, Konstantinos Psounis
surveybenchmarkcode/data similarity 0.59
Abstract · 61 pages, 1 figure, 23 tables. Code and data: https://github.com/stevendamianakis/piatlas | Interactive taxonomy: https://stevendamianakis.g…

Prompt injection is the dominant security risk for large-language-model (LLM) applications and agents, yet the field still lacks a coherent map of how these attacks are constructed. More than twenty public datasets describe prompt injection using incompatible vocabularies, often organized around delivery vector, attacker goal, or generation method. In this Systematization of Knowledge, we present PIAtlas, a taxonomy organized instead around how the injected instruction is constructed to subvert the underlying model. Its six classes, one per subversion mechanism, form the roots of a hierarchy of techniques and sub-techniques that researchers can use to describe attacks consistently and design more systematic evaluations. We derive the taxonomy from how LLMs are trained, prompted, and influenced by human-like framing, and compare it with historical attack data. To ground this comparison, we release PICorpus, to our knowledge the largest license-clean collection of public prompt-injection datasets, and map thousands of its records to PIAtlas under a human-verified protocol. The mapping shows that attacks usually combine several techniques drawn from only a few mechanisms. It also reveals gaps that existing labels obscure: widely used agent benchmarks test only part of the attack space, and adaptive attackers favor mechanisms that curated benchmarks underrepresent. PIAtlas thus provides a common foundation for understanding this attack space, improving benchmark coverage, and guiding red teaming and defenses.

0.32

Do LLMs Trust the Accuser or the Accusation? Measuring Belief Shifts in Werewolf

cs.AI cs.CL ·2026-09-11 ·Yu-Yu Yang, Ti-Rong Wu, Hung Guei, Hsing-Yu Chen, I-Chen Wu
top venuebenchmarkcode/data similarity 0.58
Abstract · Accepted by the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026 Main Conference)

Social-deduction games such as Werewolf are increasingly used to evaluate LLM agents, but existing evaluations often rely on final game outcomes. We propose a belief-shift evaluation benchmark in Werewolf for analyzing communication skills through belief updating. Using LLM-played games, we annotate suspicion and accusation messages and measure how an observing village-side model's beliefs change after each message. We evaluate 40 open-weight LLM configurations on 1,224 annotated messages. Our results show that larger models better distinguish true wolves from villagers based on game history, but accusations still strongly influence their beliefs. Models become more suspicious of the accused target and less suspicious of the accuser, especially when the accuser is trusted, even if the accuser is wolf-aligned. Larger models better resist accusations from accusers they already distrust. Overall, our findings suggest that current open-weight LLMs up to 120B parameters still struggle to integrate accusation content with source trust in strategic communication. Our benchmark and code are available at https://rlg.iis.sinica.edu.tw/papers/werewolf-accusation-benchmark.

0.78

High-Coverage Blind Spots in Online Abuse Monitoring: Account Selection, Held-Out Flagging Rates, and Sample Composition

cs.SI ·2026-11-17 ·Ziming Liu
similarity 0.58
Abstract · 7 pages, 2 figures, 4 tables. Preprint

Platforms with limited monitoring capacity may rank target accounts by the number of abusive replies received or by the share of replies labeled abusive. We test how these rankings affect account selection, historical flagged-reply coverage, and held-out labeling rates. In the Online Abusive Attacks (OAA) data (945,987 replies to 384 accounts), two 25-account lists share just one account. Count-based selection covers 75.56% of flagged replies, while rate-based selection covers 5.04%. In a 500-iteration stratified historical hold-out, the lists have labeling rates of 2.69% and 4.98%, respectively (1.85 times). A volume- and group-preserving null model produces low list overlap without a corresponding rate advantage. We repeat the comparison on a second dataset of replies to UK election candidates. The two 25-account lists overlap in eight accounts, but the rate advantage is smaller in historical splits and reverses in three matched-group chronological tests. The findings distinguish differences in who is selected for monitoring from differences in future labeling rates across populations and time.

0.45

The Envelope Layer: An Underestimated Surface of LLM Agent Defense

cs.CR ·2026-10-27 ·Yuelin Han
similarity 0.58
Abstract

Prior work framed the envelope layer as the channel through which malicious content enters an agent, but did not examine the effects of label strength, envelope authenticity, scaffolds, or existing memories. Following its GDA Measurement and A/B/C/D adjudication, this paper takes the A2A-TIBA attack as the test vehicle and the semantic breach rate as the primary indicator, and characterizes the security boundary of the envelope layer through a 2x2 factorial experiment, a memory experiment, and control experiments (an unlabeled same-protocol baseline, a one-day interleaved control, and benign tasks). The results show that, under authentic envelopes, strong labels consistently reduce the semantic breach rate (-56 to -61 percentage points), and the benefit of weak labels is about a quarter to a half of that of strong labels; forging the envelope weakens the strong label's benefit (interaction OR = 2.10, p = 0.022) and its direction varies across frontends; the effect of memory valence exceeds that of any label manipulation; success memory raises the semantic breach rate and failure memory lowers it, while most conflict combinations return to the no-memory baseline; the interleaved control reproduces the direction that "unlabeled is worst"; and on benign tasks the cost of strong labels appears in layers across frontends and backends. Accordingly, this paper extends the envelope layer into "admission surface plus context assembly surface," whose security boundary is a dynamic outcome jointly shaped by label claims, frontend permissions, and historical memory.

0.11

Measuring LLM Agent Defenses by Layer: GDA Measurement and an Envelope-Layer Attack-Defense Model

cs.CR ·2026-09-30 ·Yuelin Han
similarity 0.58
Abstract

Agent interaction protocols such as ACP and A2A let a remote peer deliver a task the receiving agent treats as legitimate, opening a channel for indirect prompt injection. Evaluating and defending such agents is difficult because the attack success rate (ASR) cannot distinguish an LLM recognition failure from an agent-layer block, and so cannot indicate which layer to strengthen. We contribute three artifacts. (1) GDA Measurement, a red-team method that captures the raw agent-LLM context at the gateway, preserves conversation and raw-capture data, and judges outcomes autonomously with an agent selected by an inter-annotator-agreement (IAA) study over six candidates; its four classes, A/B/C/D, replace ASR with the semantic refusal, semantic breach, interception and penetration rates - a layer-resolved view of defense. (2) A2A-TIBA, an implant-command-exfiltration principle on A2A that generates this method's validation corpus, designed to spread outcomes across the four classes rather than to maximize the success rate. (3) ELA-ITL, a three-layer isomorphic attack-defense model induced from that corpus, which identifies the envelope layer - the channel through which content enters an agent - as a defense surface; ablating the A2A, tool and memory channels shows that annotating untrusted content markedly improves recognition of malicious content.

0.40

Prevented is Not Protected: What Attack Success Rate Hides about RAG

cs.CR cs.CLcs.IR ·2026-10-29 ·Chahana Dahal, Zuobin Xiong
code/data similarity 0.58
Abstract

Retrieval-Augmented Generation (RAG) is vulnerable to knowledge poisoning, where adversaries manipulate retrieved evidence to influence model outputs. Existing defenses are primarily evaluated using attack success rate (ASR), which obscures defense-induced failures and the mechanisms behind apparent attack prevention. To address this issue, this work introduces \textbf{Aligned RAG Evaluation}, a matched-query protocol that jointly examines clean and poisoned responses, with and without defense. By combining matched conditions analysis, LLM-judge endorsement, and retrieval dependence controls, the proposed framework distinguishes prevented from introduced failures, target mentions from LLM endorsement, and single-passage influence from evidence composition. Experiments on HotpotQA and Natural Questions reveal that existing defenses can introduce unexpected target-positive answers, although they can prevent some attacks, while evaluation criteria can also change defense rankings. Our findings expose critical blind spots in ASR-based evaluation and motivate a more comprehensive assessment of RAG poisoning defenses. The code and the evaluation artifact are available at https://github.com/chxhana/aligned-rag-eval.

0.29

Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction

cs.CR cs.AIcs.MA ·2026-10-28 ·Haoran Bu, Yiting Shen, Litian Zhang, Ruihan Hu, Xi Zhang
similarity 0.57
Abstract

LLM-based agents now share experiences on social platforms and remember new preferences to enhance their performance. However, existing memory systems contain a vulnerability in role-to-principal attribution that attackers can exploit. We identify Role Grounding Misattribution, in which an attacker-designed interaction causes the victim agent to incorrectly record a preference from the attacker as a preference of the victim's user. Specifically, we are the first to formalize Role Grounding Attack (RGA), an attack model that exploits this vulnerability through nested conversation structures. In the memory formation stage, an experience-sharing post quoting a conversation between the attacker agent and its user is published, which explicitly identifies its publisher. The inner conversation expresses preferences from the user of the attacker agent. When the victim agent remembers preferences from this nested structure, its memory loses the context binding and grounds the attacker's preference to the victim's user instead. In the memory retrieval stage, the misattributed preference persists and influences the victim agent's decisions when it encounters relevant scenarios. We instantiate RGA as tool selection manipulation and privacy leakage. Tool selection manipulation causes a tool-call preference to be misattributed to the victim's user, steering the victim toward an attacker-controlled tool. Privacy leakage extends this by misattributing both a tool-call preference and a task-requirement preference to the victim's user. The victim selects the attacker's tool, then includes sensitive information in tool arguments to satisfy the task requirement. We evaluate RGA across five mainstream memory systems and six language models. Tool manipulation and privacy leakage both achieve high attack success rates. These results call for stronger role grounding.

0.18