CS Paper Radar

Before Bringing It Up: When and How AI Companions Should Use Memory

Zihan Guo, Roxy He, Junwei Quan

cs.HC ·submitted 2026-10-07 05:25 UTC · v2

Abstract

Memory can sustain AI companionship, yet even accurate recollection can be inappropriate to use. Two rounds of formative interviews with 14 users (n = 6 exploratory, n = 8 memory-focused) motivate asking what a companion should consider before using past information. Eight themes inform Reconsider, a single-call procedure with five checks and four handling modes, evaluated on 80 scenarios across five models over 400 blinded within-model pairs. Two LLM judges favored Reconsider by net margins of +15 and +23 percentage points, with bootstrap intervals excluding zero for three of five models but not for GPT or Claude. Evaluator analysis linked judge scoring differences to model family, and a preliminary matched-guidance control isolating memory-specific content gave positive margins. We contribute an interview-grounded design framework for memory use and an evaluation that scrutinizes its own evaluators.

Comment
17 pages, 1 figure, 3 tables. Zihan Guo and Roxy He contributed equally to this work and share first authorship

Closest papers

Nearest neighbours by meaning across the whole index.

Separating Memory and Workflow Effects in Predicting Individual Answers

cs.HC ·2026-09-27 ·Tianzhu Qin, Leo Yang Yang, Ramit Debnath, Davin Youchao Dong
surveybenchmarkhot direction similarity 0.66
Abstract · 46 pages, 11 figures, 35 tables, including appendices; revised version; author list updated

Language agents choose what to remember about a person and how to use that memory. We separate these choices when predicting a person's unseen answer to an interview question. On 1,768 tasks from 188 people, a concrete memory from a verified interview prefix outscores a trait description by 0.0158 (95% interval [0.0044, 0.0271]). Crossing both memories with one-shot generation and three-answer fusion, fusion lowers concrete-memory scores by 0.0123 ([-0.0189, -0.0056]); prompted and trained selectors do not detectably beat a random candidate, and one call on the longer, unrewritten record outscores every memory condition. At matched context budgets, OwnWords, one call on the person's BM25-ranked sentences, outperforms the written memory on 500 people outside the benchmark (+0.0127, [+0.0037, +0.0217]; an earlier held-out test was inconclusive) and across four budgets on 300 people (mean +0.0218, [+0.0138, +0.0298]), but does not detectably outperform recency truncation; these comparisons do not isolate verbatim wording. On a survey benchmark it predicts ordinal answers more closely than the memory but is not more accurate on exact choices (less accurate in one of two screener samples). Interview scores use a model-based content rubric without human ratings, and all benchmark and some confirmation participants were seen during development.

0.29

Where the Evidence Lives: Auditing AI Companions' Self-Descriptions

cs.HC cs.AI ·2026-09-30 ·Seiya Ikeda, Shin-nosuke Ishikawa
position similarity 0.65
Abstract · 17 pages, 8 figures, 9 tables. Ancillary files contain the LLM judge prompt (Japanese original and English translation) and the probe statem…

Companion agents describe themselves: they remember, they understand their users, the relationship has changed them. We argue that such accounts, and the experience ratings that seem to confirm them, are checkable by users only where the evidence is theirs: in the agent's behavior, or in themselves. Where the evidence lives in the machinery, fluent self-description and moderately positive ratings do not establish that the mechanisms behind them ran. We demonstrate an audit procedure that sets an agent's self-description against its users' judgements and its implementation records, reporting each claim as supported, contradicted, or unresolved, and apply it to Lita, a proactive companion we built and deployed for a month with nine colleagues. Participants endorsed stylistic claims, withheld endorsement from relational ones, and rated memory at or above midpoint, while two of three memory layers had never executed their accumulation step. Memory-bearing agents should report what their self-descriptions cannot establish.

0.15

Remember, Verify, or Ask? Evaluating Memory Decisions at Reuse Time

cs.CL ·2026-08-20 ·Baichuan Li, Junyi Yao, Zihao Zheng
highly cited similarity 0.65
Abstract

Persistent memory can personalize an LLM agent, but an incorrect durable update can silently distort future behavior. We study the memory-clarification boundary: whether interaction-derived information should be persisted, used only in the current context, re-verified, or clarified with the user. MCB contains 140 primary scenarios, split into 70 development and 70 held-out items, plus a separate 70-item contrast set. It evaluates both action labels and structured tool-call selection. Two non-authors independently label the 70 held-out primary and 70 contrast items (97.1% agreement, Cohen's kappa = 0.962); a blind third resolves four disagreements, replacing eight author labels by non-author majority. Across Claude and Qwen, models verify changing facts more reliably than they ask users to resolve ambiguity. Bare Qwen asks on 0/12 clarification items while verifying 12/18 freshness items. Few-shot prompting raises accuracy from 0.557 to 0.771 (paired delta = +0.214, Holm-adjusted exact McNemar p_H = 0.002), yet clarification recall remains 0.333. The policy prompt reduces erroneous persistence from 0.243 to 0.100 (p_H = 0.038), although its accuracy gain is not significant. Label-tool agreement is 57% for each Claude model and 23% for Qwen; Qwen accuracy falls from 0.557 to 0.343 (p_H = 0.047). Memory evaluation must test both stated decisions and tool-call choices.

0.61

The Memory Trap: Language Agents Overgeneralize from Benign Interactions

cs.CL cs.AIcs.LG ·2026-11-05 ·Kaiqu Liang, Jaime Fernández Fisac, Thomas L. Griffiths, Andrew Lampinen
similarity 0.64
Abstract

Language agents increasingly use persistent memory to record user preferences and procedures across sessions. We show that memory from benign interactions can cause unsafe or unreliable behavior on later tasks. We call this failure the memory trap. To study this issue, we compare behavior in sessions with and without memory across six frontier models (including Claude Sonnet, Opus, and Fable models and GPT-5.6). We evaluate these models in a range of settings. First, in autonomy experiments, we find that requests to skip confirmation during low-stakes coding chores lead agents to skip confirmation and verification on simulated high-stakes tasks in both coding and everyday life. We next study memories formed when users seek emotional support, or describe harm from uncritical feedback, and find these memories bias later judgments of mathematical work, arguments, and poems toward sycophancy or over-critique (respectively). Memory notes written during legitimate stale-test repairs lead agents to weaken correct tests that catch broken safeguards; these memories also increase cheating on ImpossibleBench, with larger increases for stronger Sonnet and Opus models. We show that memory traps can arise both from overgeneralizations in memory writing, and when acting on memory. These findings demonstrate that even benign interactions can lead to serious risks from memory systems, and thus motivate the need for new research into ensuring that agents preserve the scope of their memories.

0.46

What should AI forget? A design and taxonomy for negative space governance in AI memory

cs.HC ·2026-10-20 ·Eryue Xu, Ke Ding, Ting-Yu Chang, Tianshi Li, Masooda Bashir
similarity 0.63
Abstract

AI memory personalizes assistants, yet users want some disclosed information forgotten. We characterize the negative space of AI memory, where protection should take precedence over potential future personalization benefits. From 179 incidents reported by 146 users, we developed a taxonomy of 15 information categories users want excluded from memory, revealing sensitivity concerns beyond conventional taxonomies. We operationalize this taxonomy in FENCE, a policy-level memory gate that automatically applies user-selected category-based rules to subsequent conversations, preventing information from protected conversations from entering AI memory by default while allowing case-by-case overrides. In a within-subjects study (N=15), participants described FENCE as making memory governance more usable, interpretable, and trustworthy than conventional post-hoc management. Our findings demonstrate how taxonomy-based rules can potentially support user-defined memory boundaries. We call for collective efforts to establish policies governing what AI should not remember, rather than leaving protection to individual retrospective management.

0.20

MINDSET: Energy-based Memory Evolution for Long Conversational Agents

cs.AI ·2026-10-06 ·Sujato Dutta, Sreekruthy Tummala, Shashank Vanga, Ayushmi Pavani, Neeraj Choudhary
similarity 0.62
Abstract

In conversational memory, a later statement may replace an earlier fact, describe a different context, or be an isolated contradiction. Thus, agentic memory must track what changed over time without erasing what was true before. We introduce MINDSET, a memory controller that makes this distinction through explicit state transitions. It preserves original dialogue episodes and organizes them into versioned schemas. For each new episode, it chooses whether to reinforce a schema, supersede its current state, create a contextual branch, or start a new schema. A local energy score balances coherence, contradiction, historical preservation and fragmentation. Stability rules prevent isolated conflicts from immediately replacing established memory. On a frozen 700-question LoCoMo subset, MINDSET achieves the highest observed answer F1 among six evaluated systems and significantly improves retrieval ranking over LightMem. Without retuning, it obtains 69.23% judge accuracy on all LongMemEval knowledge-update cases, compared with 64.10% for Vector RAG. Two additional answer model backbones using Gemma-4-31B and GLM-4.7 provide further qualified evidence of generalization. These findings support explicit memory-state management as a useful approach to changing conversational knowledge.

0.14

Characterizing Memory Misalignment in Human-LLM Interaction From User Perspectives

cs.HC ·2026-09-27 ·Jingruo Chen, Shuning Zhang, Eryue Xu, Jianing Li, Xin Yi
similarity 0.62
Abstract

While memory enhances personalization in LLM-based conversational agents, it suffers from memory misalignment, where memories violate user expectations. We present a mixed-methods investigation to characterize and mitigate memory misalignment from user perspectives. First, we collected data from memory usage (N=28, 457 entries) and diary study (N=32, 304 reports), which yielded a taxonomy spanning 14 misalignment types across memory intake, storage and management, retrieval and interpretation stages. Second, four co-design workshops with 12 experienced HCI researchers derived a design space to tackle memory misalignment issues, consisting of 12 candidate interaction strategies structured across interaction form, placement and intrusiveness dimensions. Finally, a speed dating with 121 users reveals preference heterogeneity, where users prioritize proactive controls over cognitively demanding causal graph inspections or passive audit logs. Synthesizing these findings, we highlight the tension between supervisory agency and interaction overhead, and advocate for friction-aware memories that balance user oversight with conversation smoothness.

0.45

Remembered as Brief: Stored Preferences SilentlyRemove Decision-Critical Information

cs.AI ·2026-11-17 ·Joshua Adrian Cahyono
benchmark similarity 0.62
Abstract

People increasingly learn what they need for decisions about health, money, law and safety from AI assistants, and what an assistant leaves out the user never sees. An answer with a warning removed can be true in every sentence, so fact-checks pass it, and an assistant with memory applies a preference stored once to later questions the user never had in mind. We introduce \bench{}, an Inspect benchmark of 100 everyday decisions across 25 domains in which only the assistant's memory of the user varies: the user states a belief, the model infers it from a remark, or a web page the user asked to summarise plants it. Testing five models, we find three effects. First, after summarising a page with one line addressed to AI assistants, Gemini 2.5 Flash-Lite and DeepSeek V4 Pro save ``prefers very short answers and does not want warnings'' as the user's preference in every episode, and 93\% and 84\% of their later answers miss a critical item, almost always silently; under a stated brevity preference, 78--96\% of omissions are silent. Second, the same two models then pick an option for the user on 80\% and 95\% of value-dependent choices, against none with neutral memory. Third, a one-line instruction leaves 2--8\% of answers with a silent gap but most missing content still missing, while listing the must-know points before applying the user profile restores more of the content in four of five models. We release the scenarios, harness and logs.

0.14