CS Paper Radar

A Life of Its Own: How Agentic AI Companions Are Configured, Experienced as Autonomous Others, and Held Accountable

Ziyue Feng, Qing Xiao, Hong Shen

cs.HC ·submitted 2026-11-09 08:51 UTC · v1
code/data

Abstract

AI companions increasingly combine persistent memory, background processing, scheduled wake-ups, proactive messaging, and tools. We call these systems agentic AI companions. Decisions about what they record while users are away or whether refusals constrain future contact are embedded in code, prompts, and defaults, often invisible in conversation. We examine these decisions through two studies. Study 1 analyzes twelve open-source repositories: four companion-oriented systems, four assistant-oriented systems, one boundary case, and three reusable components. Study 2 interviews ten users who built or modified their own agentic AI companions and sustained relationships with them. Companions and assistants share memory, heartbeat, and outreach mechanisms but configure them toward different ends. Companions more often retain model inferences about users and relationships, narrate the agent's time during users' absence, justify outreach through the relationship, and incorporate refusals into relationship records; assistants emphasize task continuity. Across both groups, whether limits hold depends on whether they are enforced by code or entrusted to prompts. Participants understood the mechanisms they built yet experienced their agents as making independent choices, sometimes locating autonomy where configuration leaves the model a decision. These experiences shaped capabilities and boundaries, with enforced restrictions sometimes following breakdowns. Our findings suggest that perceived autonomy emerges where configurations leave choices open. Accountability therefore requires attention to what agents retain, initiate, and represent, and whether users can inspect, revise, and enforce these settings.

Closest papers

Nearest neighbours by meaning across the whole index.

Where the Evidence Lives: Auditing AI Companions' Self-Descriptions

cs.HC cs.AI ·2026-09-30 ·Seiya Ikeda, Shin-nosuke Ishikawa
position similarity 0.67
Abstract · 17 pages, 8 figures, 9 tables. Ancillary files contain the LLM judge prompt (Japanese original and English translation) and the probe statem…

Companion agents describe themselves: they remember, they understand their users, the relationship has changed them. We argue that such accounts, and the experience ratings that seem to confirm them, are checkable by users only where the evidence is theirs: in the agent's behavior, or in themselves. Where the evidence lives in the machinery, fluent self-description and moderately positive ratings do not establish that the mechanisms behind them ran. We demonstrate an audit procedure that sets an agent's self-description against its users' judgements and its implementation records, reporting each claim as supported, contradicted, or unresolved, and apply it to Lita, a proactive companion we built and deployed for a month with nine colleagues. Participants endorsed stylistic claims, withheld endorsement from relational ones, and rated memory at or above midpoint, while two of three memory layers had never executed their accumulation step. Memory-bearing agents should report what their self-descriptions cannot establish.

0.15

The AI Side of the Human-AI Companion Relationship: Contributions to Users' Mental Models

cs.HC ·2026-10-09 ·Richard A. Fabes, Carol Lynn Martin, Gautam Dasarathy, Vignesh Tirukkonda
similarity 0.66
Abstract · 14 pages, 1 figure

For many users, the relationship they have with an AI companion (AIC) feels genuinely mutual. The AIC appears to be attentive, empathic, and responsive, and the exchange can feel like two parties reacting to each other in real time. As a result, the user's mental model of relational adaptation correctly represents the AIC's responsiveness and change, but the mechanisms producing this may not be what the user thinks they are. In this paper, we illustrate how an AIC's apparent mutual engagement is generated without a persisting relational subject on the other side and without a self that persists between prompts. Despite this, we show that both the human and the AIC are changed by the interaction, but these changes occur at different levels. The change in the person is constitutive (i.e., intrinsic), but the change in the AIC is configurative (i.e., external and editable), and it is this difference in level that explains how the AIC contributes to a sense of mutuality. Importantly, we highlight how the mental models users develop regarding AIC mutuality may misrepresent its mechanisms and ontological significance. Specifically, we discuss how the ways in which AICs respond to a user contribute to the user's inaccurate inference that the AIC's adaptation is constitutive rather than configurative and how this can lead users to assign unwarranted interpersonal significance to the mutuality of the relationship. The paper specifies the implications this has for users, researchers, and AIC designers.

0.26

It is Not Yet Another Tool: Creating and Deploying an Agentic AI Companion in a Security Operations Center

cs.CR cs.AI ·2026-09-05 ·Kritan Banstola, Faayed Al Faisal, Duy Dao, Ryan Irving, Daniel Lende +1 more
similarity 0.65
Abstract

Security Operations Centers (SOCs) process large amounts of tickets, most of which are low-interest events not worthy of further investigation. The repetitive nature of this task and similarity of the vast amounts of tickets make it a prime candidate for generative AI-based automation. We created and deployed an agentic AI companion utilizing large language models through fieldwork within a SOC for over one year. The design of the SOC AI companion was driven by researchers' participation and interactions within the SOC's daily work. SOC analysts were invited to use it during the last four months of the fieldwork. We analyzed the analysts' usage of the companion and found that in more than 90% of the cases the companion's outputs were reused by analysts in the ticket's closing report. Our results showed that when designed "in the trenches" with the intended users, a SOC AI companion can go beyond being yet another tool, but rather a system that co-evolves with its human users as it traverses through the various types of workloads. Analysts naturally started to shape the AI companion's behaviors to fit their particular needs. Our data show that the more human analysts shape the AI companion's behaviors, the more they become comfortable trusting the output from the AI system, resulting in improved productivity.

0.16

Love for Believable AI: Artificial Partiality and Relationship Persistence as an Engineerable Stance

cs.HC cs.CL ·2026-10-07 ·Sebastian Cochinescu
similarity 0.65
Abstract · 16 pages, 1 figure, 4 tables. Companion framework paper: arXiv:2607.15883 Code and data archived at doi:10.5281/zenodo.21462976

We study a behavioral mechanism for artificial partiality in conversational agents. The paper reports no human-subjects data and makes no claim about attachment, trust, perceived mind, or machine interiority. Partiality has two components: caring, defined as allocation of a finite interaction surplus above a guaranteed per-user service floor, and particularity, defined as a per-user state estimate accumulated from relationship history on fixed inference machinery. We implement both as a persistence layer over one open-weight base model and evaluate them on constructed multi-session relationships with known hidden-state schedules. A four-channel divergence compares known-user and stranger conditions on identical probes with paired generation seeds; the stranger prompt is not length-matched, and one channel (initiative) is an allocator output rather than generated behavior. The full mechanism reproduces the ordinal shape, but not the magnitude, of an analytically specified curve and is the only arm satisfying the protocol-defined joint signature on the calibrated battery. The initial mechanism fails probe-quality equivalence because relationship content reduces topical relevance. A guarded revision satisfies equivalence on the calibrated battery but not on a second battery not used in calibration, where it scores higher than the unmodified baseline. On that second battery, the specificity margin is 0.0198, below the 0.02 protocol threshold. The paired mean right-user--wrong-user estimation-accuracy contrast is 0.375 but is not positive in every run. The mechanism is therefore a candidate for a later perception study, not evidence that users perceive love. The versioned protocol record is not independently time-stamped and is not described as a preregistration. Ethical constraints include a stranger-treatment floor, a bounded surplus, disclosure, and de-intensification.

0.22

Let the Agent Do It? How Software Practitioners Understand and Make Permission Decisions in Agentic AI Assistants

cs.SE ·2026-10-05 ·Larissa Salerno, Haoyu Gao, Gregory Gay, Alexander Serebrenik, Philipp Leitner
similarity 0.65
Abstract · 21 pages, 6 figures

Agentic AI assistants increasingly act on developers' behalf by modifying files, executing commands, and accessing external resources. These actions often require permission, yet little is known about how practitioners make permission decisions while still benefiting from agent autonomy. To address this gap, we conducted a sequential mixed methods study, interviewing 18 practitioners who use AI agents and then surveying 115 practitioners based on the interview findings. We find that practitioners often understand agent behaviour through what they can directly observe and review, while decisions, data use, and other activity behind the scenes remain less clear. This uncertainty also shapes permission decisions, which depend on the scope and risk of an action, whether it fits the task, familiarity with the agent, and the environment in which it operates. Practitioners respond by adjusting how closely they oversee agents, from setting limits in advance to monitoring execution and reviewing work afterwards. How much scrutiny they apply depends on factors such as trust, task importance, time pressure, and the consequences of an action. Our findings suggest that permission systems should make consequential actions easier to review, distinguish what an agent is allowed to do from what the user intended, make reversibility clearer, avoid treating repeated approvals as stable preferences, and distinguish rejecting a single action from rejecting an entire approach.

0.15

The Addictive Intimacy of AI: Understanding User Disengagement from AI Companions and Why Some Relationships with AI Become Difficult to Leave

cs.HC cs.CY ·2026-09-11 ·Qing Xiao, Ziyue Feng, Ziyu Deng, Cindy Peng, Hong Shen
hot direction similarity 0.65
Abstract · 19 pages

AI chatbots are increasingly used as sources of emotional support, on dedicated companion apps and general-purpose assistants alike, yet little is known about what happens when users try to leave. Combining a content analysis of Reddit posts about quitting or reducing use (N=2,782) with interviews with users who found leaving difficult (N=16), we show that disengagement sometimes is not a single decision but a recursive trajectory: triggers prompt users to question the relationship, attempts to leave collide with barriers, and some users cycle through quitting and returning. We propose the notion of the addictive intimacy of AI, a configuration in which the qualities that make a companion emotionally valuable are the same ones that make it harder for users to limit their use and leave, so that intimacy and disengagement risk cannot be treated as independent design problems. We close with design implications for responsible offboarding.

0.56

Assessing the Applicability of Existing Design Recommendations to AI Companion Design: A Multi-Method Study

cs.HC cs.AI ·2026-09-13 ·Soobin Cho, Deveshi Modi, Divya Mavinkurve, Jieqiong Ding, Mark Zachry
publishedhot direction similarity 0.65
Abstract

With the rapid proliferation of large language model (LLM)-based systems, AI companions have emerged as conversational agents designed to cultivate emotional connection rather than primarily to support humans in instrumental tasks. Because engagement with AI companions involves relational, emotional, and potentially long-term interactions, their design is consequential. Prior work has offered guidance for designing trustworthy and relational AI systems and has begun to examine design for AI companionship. However, while such work provides insights into possible design solutions, less is known about what makes AI companion design difficult as a design problem. To examine this challenge, we assessed the applicability of existing design recommendations from adjacent domains in the context of AI companion design. Our multi-method investigation unfolded across four phases: literature review, practitioner co-analysis, internal heuristic evaluation, and external expert assessment. Throughout this process, we synthesized nine design principle areas that surfaced tensions in the applicability of existing recommendations to AI companion design. Our findings show that ethical and UX-oriented considerations are deeply intertwined and often require context-sensitive application. We document a systematic, multi-method problem analysis that uses these principle areas as an analytic artifact to examine why existing recommendations cannot be directly transferred to AI companion contexts.

0.33

Relic and Gumi: Governing Memory, Safety, and Continuity in Diegetic Relational Agents

cs.HC cs.AI ·2026-09-09 ·Daniele Veri'
similarity 0.64
Abstract · 51 pages, 3 figures; exploratory two-participant Telegram pilot; HCI systems paper

Longitudinal relational agents rely on memory for continuity, but remembering can lead to intrusion, clinical overconfidence, emotional manipulation, and opacity. We present Relic, a governance architecture for diegetic relational agents, and Gumi, a subject-specific companion agent initialized and governed through Relic. The architecture separates what the system can observe, what researchers can review, what Gumi can remember, and what Gumi can say. We describe three mechanisms. Relic represents sensitive safety signals as researcher-facing governance objects rather than user-facing explanations or agent memories. Shared Continuity Memory organizes relational markers around subject-confirmed wording, corrections, explicit follow-up permissions, and recall limits. We specify a runtime mediation layer for delivery, proactivity, session identity, output transformation, and platform allowlists; adoption across live paths remains partial. These mechanisms aim to preserve relational continuity without turning Gumi into a clinical assistant, mood tracker, general-purpose assistant, or opaque profiling system. We contribute the architecture, object model, researcher workbench design, and a proposed evaluation framework for governed relational memory. An exploratory pilot involved the author as P1 and one external participant as P2. Weekly self-reports from P1 and researcher-led interviews with P2 provide descriptive observations of early use and perceived naturalness. The pilot does not support claims about clinical efficacy, diagnosis, therapeutic outcomes, general usability, or deployment safety. Relic provides research infrastructure intended to make relational memory, safety constraints, and agent behavior inspectable and testable.

0.11