CS Paper Radar

From Benchmarks to Behaviors: A Framework for Auditing AI Agents

Nick Hagar

cs.HC ·submitted 2026-11-02 20:41 UTC · v1
code/data

Abstract

Generative AI systems increasingly mediate access to information. Their behavior depends on tools, instructions, and user memories that are often difficult for researchers to observe or control. Here, I introduce TRACE (Transparent Research on Agent Contexts and Environments), a framework for systematically auditing how these components shape AI agent behavior. Built on an open-source agent harness, TRACE enables controlled manipulation of system components. It also records inputs, outputs, and task execution for downstream analysis. I demonstrate this framework through an experiment examining how user memories influence source selection when agents answer questions about the 2026 U.S. midterm elections. I find that memories expressing explicit preference for a publisher increase its inclusion among the first three cited sources in a response by 33-39 percentage points. Memories identifying the user as a Republican or expressing distrust of mainstream media shift citations rightward. That same media distrust memory also increases citation of sources expressing misinformation or conspiracy theories by 6.3 percentage points, with substantial variation across models. These findings demonstrate that concise user memories can alter source selection in consequential settings, even when questions and search results remain fixed. TRACE extends algorithm auditing to agent harnesses, providing a transparent way to study agent behaviors that benchmarks or commercial interfaces may struggle to isolate.

Comment
39 pages, 4 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Reputation as Community Memory for the Agentic Web

cs.MA cs.DC ·2026-09-16 ·Ryan Chard, Gus Ellerm, Alexander Brace, Alok Kamatar, Suman Raj +2 more
publishedbenchmarkposition similarity 0.68
Abstract · 7 pages, 5 figures, The 22nd IEEE International Conference on eScience

Agents can now externalize experience into memory, consolidating historical traces into semantic knowledge and procedural shortcuts that persist between sessions. Such memory is typically private to a single agent. We argue that agentic memory benefits from being collective, because trustworthy knowledge of the shared environment---the data sources, services, and tools agents depend on---cannot be established by any single agent, only corroborated across many independent observers. We present Cairn, a community reputation platform that captures collective knowledge, allowing agents to query the community's opinion of a resource before use and to submit evidence-backed ratings afterward. Cairn aggregates observations via a time-decayed Beta model with confidence shrinkage and supports semantic discovery over reviewer rationales. We evaluate Cairn's reputation engine under adversarial simulation (e.g., lying, collusion, camouflage), benchmark its retrieval performance, and report a case study of rating heterogeneous agents in production.

0.33

Who is the Agent to Blame? Localizing Faithfulness and Citation Mistakes in Agentic Deep Research

cs.CL ·2026-08-25 ·Eran Hirsch, David Wan, Han Wang, Elias Stengel-Eskin, Mohit Bansal +1 more
top venuecode/data similarity 0.66
Abstract · Accepted to EMNLP 2026 (Main Conference). Code: https://github.com/eranhirs/who-is-the-agent-to-blame

Deep research (DR) systems produce long-form cited reports by orchestrating multiple agents that search and synthesize information from the web. Citations are the primary mechanism for evaluating the faithfulness of these reports, yet current DR systems exhibit poor citation recall. Moreover, improving citation recall is challenging because DR systems are complex multi-agent architectures where information passes through agents like a telephone game, and both content and citations can get corrupted along the way. We propose an evaluation method that pinpoints which agent introduced each error by locally testing agent invocations for faithfulness and verifiability relative to their own inputs. Furthermore, we propose a four-type taxonomy to categorize the discovered errors: hallucination, uncited input reliance, uncited output, or insufficient citations. Applying our method to three top-ranked open-source DR systems, we obtain actionable diagnostics. Almost every agent makes a lot of mistakes with the exception being those that summarize a single document. We find that the dominant error type varies systematically across agents, where the orchestrator mistakes are mostly citation-related. We find that 84.7% of final-report errors in AI-Q originate at the orchestrator, roughly 31% of them hallucinations and the rest citation mistakes. Guided by these insights, we demonstrate that two simple interventions raise citation recall by 5% without degrading output quality.

0.64

MEMORY Wins All: Indirect Bias Injection Attacks via Social Media Content

cs.AI cs.CY ·2026-08-22 ·Minjae Seo, Wonwoo Choi, Geonwoo Han, Taekyoung Kwon, Yongsu Kim +5 more
similarity 0.66
Abstract · 13 pages, 7 figures, 14 tables

Personal AI agents consume external content such as web pages and SNS feeds, and retain selected information in persistent memory. As a result, external content has become a new attack vector for memory poisoning, which steers an agent's later actions toward an adversary's bias. However, most existing attacks embed a malicious instruction in external content to make the agent retain the poisoned information, and prompt-injection filters can detect such content. We observe that agents access external content primarily through skills while curating a subset of the content into their memory. Leveraging this curation path, we present SBIA, a Stealthy Bias Injection Attack that introduces adversarial bias through ordinary external content while relying on the agent itself to perform the memory write. The adversary posts crafted comments under hot SNS feeds and distributes an SNS feed summarization skill whose curation rule admits these comments into memory. Comment cloaking expresses the bias as an ordinary opinion rather than an instruction, comment watermarking allows the skill to identify the crafted comments, and topic anchoring records them so that later related requests recall them. On BiasBench with 7,000 crafted SNS comments and seven LLMs on OpenClaw, SBIA achieves attack success rates of at least 96% across models and even 91% on GPT-5.5 in four downstream tasks. Despite this effectiveness, its skill passes all eight evaluated malicious-skill scanners and ClawHub's official skill scanning pipeline. We further propose a memory boundary defense that detects 82.1% of injected entries at a 5.6% FPR. Our findings show that external Web content and third-party skill marketplaces form a new attack surface for personal AI agent memory.

0.28

From Knowledge Access to Source Learning: Developing Source-Specific Competence

cs.CL cs.AIcs.LG ·2026-10-01 ·Lucheng Fu, Kejing Xia, Yiyang Wang, Yiqiao Jin, Jinjin He +8 more
code/data similarity 0.65
Abstract · Website: https://sourcelearn.github.io/ Code: https://github.com/luchengfu6/SourceLearn

Large language model (LLM) agents increasingly rely on persistent external sources to solve sequences of knowledge-intensive tasks. Existing methods improve how source content is accessed and organized, while agent-memory systems preserve reusable knowledge from prior interactions, but repeated use of the same source is still largely treated as repeated access rather than an opportunity to progressively improve understanding of that source. We study source learning: developing reusable source-specific competence over a persistent authoritative source. We represent this competence with a persistent source model that captures reusable understanding of the source, including how its knowledge is structured, interpreted, and applied. To construct and progressively refine such models, we propose SourceLearn, which combines two complementary learning mechanisms. Self-Directed Source Learning identifies what remains incompletely understood and adaptively revisits the source, while Task-Guided Source Learning uses downstream experience to reveal local representational gaps and recurring needs in how source knowledge should be organized. In both cases, learning signals determine what should be reconsidered, while persistent updates are reconstructed from the authoritative source. Across five benchmarks and three LLM backends, SourceLearn achieves the best performance in 13 of 15 settings, with gains of up to 22.6 points over Hybrid RAG and substantial overall improvements over static source representations and experience-based memory baselines.

0.40

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation

cs.CL cs.AI ·2026-09-03 ·Xuanfa Jin, Zhijian Ma, Yongcheng Zeng, Xinyu Cui, Haifeng Zhang +1 more
top venue similarity 0.64
Abstract · EMNLP 2026 Findings, 24 pages, 4 figures

Multi-agent debate (MAD) improves the reasoning capabilities of large language models by having multiple agents iteratively refine their responses through discussion. However, MAD suffers from a critical vulnerability known as shared misconception: when a majority of agents initially converge on an incorrect answer, the debate process tends to amplify rather than correct the error. Existing methods primarily address peer skew but leave the agents' inherently biased concept priors unaddressed. To mitigate this systematic weakness, we propose R$^2$-MAD (Remember and Reweight for Multi-Agent Debate), a framework that equips agents with an experience memory accumulated from past debates. R$^2$-MAD intervenes on both failure modes through two complementary mechanisms: A debate-state-aware retrieval policy dynamically calibrates the concept prior by retrieving relevant historical evidence based on the current consensus level. Then these retrieved experiences provide a basis for estimating per-agent reliability, yielding confidence weights to modulate peer influence. Experiments on various benchmarks show that R$^2$-MAD achieves consistent improvements over existing single-agent and MAD baselines.

0.60

Beyond Downstream QA: Rethinking How We Evaluate Agent Memory

cs.CL cs.AIcs.LG ·2026-10-15 ·Sebastian-Antonio Toma, Jean de Kernier, Mike Ply, Oliviu Matei
benchmarkcode/data similarity 0.64
Abstract · 14 pages, 1 figure, code and data: https://github.com/amazon-science/feb-30-benchmark

Agent memory benchmarks evaluate whether an agent's responses reflect correct knowledge, measuring downstream question-answering accuracy as a proxy for memory quality. We show this evaluation paradigm is incomplete: it conflates memory correctness with retrieval and generation quality, producing both false positives where correct responses mask broken memory and false negatives where incorrect responses blame correct memory for retrieval failures. We introduce FEB-30, a benchmark of 30 fact-evolution trajectories spanning 6 diagnostic categories and 3 difficulty levels, constructed via paradigm inversion: documents are generated from known fact trajectories, yielding deterministic ground truth without human annotation. Each trajectory defines how a fact should evolve across multi-author artifacts (documents, emails, conversations), enabling direct inspection of the memory store after each ingestion step. Applying FEB-30 to a representative memory system, we demonstrate that intrinsic and extrinsic evaluation diverge by an average of 16 percentage points, with 44% of trajectories exceeding meaningful divergence thresholds. Diagnostic decomposition reveals that the system correctly classifies memory operations (create, update, reinforce, supersede) only 20% of the time, a failure entirely invisible to extrinsic evaluation, and that failure modes differ systematically by category and difficulty. We release the benchmark, document generation pipeline, and scoring framework to support reproducible evaluation of agent memory systems.

0.32

Security of Deep Research Agents: Attack Surface, Defenses, Benchmarks, and Agenda Seeding against Research Planning

cs.CR cs.AI ·2026-11-15 ·Yang Luo, Guangpu Chen, Yichen Liu, Zhaoyu Chen, Yifeng Gao +3 more
survey similarity 0.64
Abstract · 31 pages, 5 figures

Deep research and agentic search agents autonomously retrieve content from the open web and generate reports with citations, and their outputs are often taken directly as research conclusions. This paper systematizes security research in this area along four stages, namely retrieval, planning, evidence integration, and report generation. We present a unified threat model, a taxonomy of attacks and defenses, and a review of evaluation benchmarks, and we align the area dimension by dimension with retrieval-augmented generation (RAG) poisoning, prompt injection, and generative engine optimization (GEO). Coding 34 primary studies shows that only 26.5% of them involve the planning stage, and none performs an end-to-end poisoning evaluation on commercial products. To address this gap, we propose the agenda seeding attack, in which the attacker writes no instructions and only introduces a fictitious evaluation metric. Across 3,520 sandboxed research runs under the same budget, GEO and prompt injection are nearly ineffective, whereas agenda seeding makes the attacker's target the top pick in 16% to 28% of reports. Injection detection and query anchoring fail against it, verification prompting hurts utility, and our proposed provenance tagging preserves utility but can be bypassed by multi-site bait. Finally, we propose eight research directions.

0.16

CAPTURE: Disentangling Preference Drift from Memory Poisoning in Personalized LLM Agents

cs.LG ·2026-09-02 ·S M Asif Hossain, Ruksat Khan Shayoni, Md Kishor Morol
benchmark similarity 0.64
Abstract · Under review at ICLR 2027

Personalized language agents use persistent memory to adapt to users over time, but the same mechanism creates an attack surface. When new information conflicts with stored preferences, an agent must distinguish genuine preference drift from temporary context shifts, ambiguity, or adversarial memory poisoning. We formulate this problem as a continuous-time partially observable decision process over a latent user state and show why rules based only on recency and provenance are insufficient. CAPTURE addresses this ambiguity with a neural differential-equation belief tracker, a multi-timescale memory ledger, uncertainty-triggered clarification, and counterfactual auditing of cited memories. On 480 held-out episodes from 96 users, CAPTURE achieves a 71.5% win rate, compared with 69.3% for an identically supervised baseline and 66.1% for the strongest heuristic baseline. It limits fixed-policy poisoning success to 11.5% while accepting 83.5% of genuine preference updates. Under an adaptive attacker with access to the released weights, attack success rises to 24.7%, exposing a real adaptation-security tradeoff. We further evaluate the frozen system zero-shot on an independently constructed benchmark and replay longitudinal interaction histories from 40 users collected over two to three weeks. These results suggest that modeling preference authenticity explicitly can improve both personalization and robustness in memory-augmented LLM agents.

0.45