CS Paper Radar

Judgement and the Feeling of Agency When Instructing a Computer to Act

Johanna Didion, Diego Garaialde, David Coyle

cs.HC ·submitted 2026-08-10 14:14 UTC · v1

Abstract

In cognitive psychology, the sense of agency refers to the feeling of controlling an external event through one's own actions. This paper asks how people experience agency when they give an instruction to a computer, which then acts on their behalf. Research in human-human interaction has found that while people experience agency over the initial actions taken by another in response to their command, they may not experience agency over the ultimate outcome of those actions. We present two experiments. The first draws on prior studies of human-to-human interaction but replaces a human co-actor with a computer. The second study addresses potential limitations in the first, exploring the effect of more meaningful outcomes, and assessing both pre-reflective feelings and post-reflective judgements of agency. We find that participants experienced agency over the initial action taken by the computer in response to their command, but did not experience pre-reflective agency over the final outcome. Interestingly, however, they did judge themselves to have caused the final outcome when it was meaningful and their command specific. This difference between the pre-reflective feeling and post-reflective judgement of agency has important implications for design. If designers want people to feel in control and take responsibility for outcomes that result from instructions given to a computer, they cannot rely on pre-reflective intuition. Instead, we should design systems that encourage people to reflect on their actions and make deliberative judgments regarding responsibility.

Closest papers

Nearest neighbours by meaning across the whole index.

Perceived Agency During Agent-Avatar Control Transitions in Virtual Reality

cs.HC cs.AI ·2026-11-09 ·Jingyi Zhang, Ruijun Sun, Klara Brandstatter, Nels Numan, Anthony Steed
similarity 0.66
Abstract

Prior beliefs about whether a virtual human (VH) is controlled by a human or a computer can shape how users interpret and respond to its behaviour. As emerging hybrid VH systems allow control to transfer between human operators and computer programs during an ongoing interaction, it remains unclear whether perceived agency updates with the actual controller or continues to reflect users' prior beliefs and first impressions. We investigated this question in a within-subject VR study with 24 participants across four conditions: always computer-controlled, computer-to-human, human-to-computer, and always human-controlled. We designed an interaction task in which participants acted as interviewers, asking the VHs a fixed set of questions. Before each interaction, participants were informed whether the VH was human- or computer-controlled, matching the actual controller at the beginning, but were unaware of possible control transitions. We found that perceived agency was strongly influenced by the controller presented at the beginning of the interaction, with no evidence of updating following the control transitions. Social evaluations including likeability and co-presence were generally higher in conditions where human control occurred later and covered a greater proportion of the interaction. The results suggest that prior beliefs and first impressions continued to shape perceived agency, whereas social evaluations were more sensitive to the VH's actual behaviour over the interaction.

0.10

Forced Yet Free: What Magicians' Forcing Reveals Beyond Intentional Binding

cs.HC ·2026-09-23 ·Koichi Toida
similarity 0.60
Abstract · 9 pages, 1 figure. v2: Minor theoretical clarification and an additional reference

In magicians' forcing, spectators may experience a choice as self-determined even when that choice has been externally directed. Research on the sense of agency has developed largely around action-outcome relations, most notably intentional binding; however, the problem of choice authorship (why a particular choice is experienced as originating from oneself) must be treated as distinct. This paper integrates research on agency and forcing by distinguishing the locus of intervention within the choice-action-outcome chain, self-attribution at the Decision, Action, and Outcome levels, and the processes by which feeling of agency and judgment of agency are constructed. On this basis, a distinction is drawn between intentional binding, which concerns the temporal/causal relation between action and outcome, and the different binding problem exposed by forcing: the relation between a choice and its author. This theoretical relation is termed authorship binding. Authorship binding does not denote a new implicit measure; rather, it refers to the constructive relation through which a choice whose formation has been directed by external factors can nevertheless be experienced as originating from the self. Forcing can sustain this relation not by eliminating agency, but by selectively preserving, substituting, and redistributing agency cues. XR, in which body-centred spatial relations can be manipulated, is further conceptualised as a medium for extending the forced-yet-free structure into space.

0.28

A Dual-Process Perspective on Nudge Susceptibility in LLM-Based GUI Agents

cs.AI econ.GN ·2026-09-17 ·Haya Halimeh, Sascha Kaltenpoth, Kevin Bösch, Oliver Müller
similarity 0.60
Abstract · Preprint of a manuscript completed in November, 2025

LLM-based GUI agents increasingly act on behalf of users in digital environments that were designed with human users in mind. These graphical user interfaces were designed to support, but also deliberately steer, the behaviour and decisions of users. While behavioural biases in the textual outputs of LLMs are well-documented, far less is known about how such influence operates when models act as agents that perceive interfaces and execute decisions---and, in particular, whether the reasoning capabilities increasingly built into these agents make them more robust to it. Drawing on Dual-Process Theory, we empirically investigate whether LLM-based GUI agents are susceptible to automatic (Type 1) and reflective (Type 2) digital nudges, and how their reasoning configuration moderates this susceptibility. In a randomized online shopping experiment with 3,600 agents and a total of 21,600 simulations across six frontier models from three providers, we found that agents were vulnerable to both nudge types. Crucially, the reasoning configuration moderated these effects in opposing directions, reducing susceptibility to automatic default nudges while heightening it to reflective social influence nudges. Extensive reasoning therefore did not make agents more robust but redirected the route through which choice architecture takes effect. Exploratory analysis further showed this redirection to be systematically structured by model scale. Beyond establishing nudge susceptibility as a behavioural property of agentic AI, the study positions interface design as a governance concern for organizations that delegate decisions to autonomous agents.

0.50

When Intelligence Becomes Agency: A Theory of Governed, Proactive Agency for Symbiotic AI Systems

cs.AI ·2026-09-07 ·João Dias Ferreira
benchmark similarity 0.59
Abstract · 75 pages, 3 figures, 9 tables

Persistent AI assistants are intended to extend human attention, memory, and coordination across changing digital and physical environments. To be truly useful they must do more than just act when asked. They must decide on their own whether a situation warrants behavior at all, when it does and in what mode, whether to act, ask, monitor, defer or deliberately refrain. We call this the activation problem. Research on commitment, appraisal, mixed-initiative interaction and delegation each illuminates part of it, but none ties situated activation to continuing authorization and accountability. This paper develops a conceptual and formal framework for governed proactive agency, organizing behavior across time through perception, intent, affective-conative appraisal, constraint, and feedback. It distinguishes autonomous and delegated agency and defines symbiotic agency as delegation under a standing, revocable mandate, with continuing coupling to the principal's situation, calibrated inference of their condition, and bounded personalization. The distinctive contribution is an integrated account linking activation decisions to authorized perception, behavior selection, authority containment, traceable restraint, and constrained adaptation, with behavioral episodes as the unit of analysis. Through an agency classification method, an evaluation framework, proposed benchmark scenarios, and a reference architecture, the account provides a basis for specifying and assessing whether assistance is warranted, timely, authorized, and answerable beyond task completion alone. It is intended to guide the development and evaluation of always-present personal assistants and embodied support systems that augment human capabilities while preserving the principal's authority and judgment.

0.32

Following Delegation Traces to Study Human Oversight in Interaction with Multimodal AI

cs.HC cs.AIcs.CV ·2026-10-15 ·Agnese Chiatti, Matteo Matteucci, Viola Schiaffonati
survey similarity 0.59
Abstract

As multimodal AI systems are becoming our everyday thought partners, the shift towards studying how users transfer control to these systems becomes especially urgent. In this paper, we report an overview of findings from an online study with n=56 AI-literate participants and 392 Video Question Answering trials. After inspecting each multimodal AI response, participants rated task-specific trust and selected one of three intended policies for an analogous future task: reject AI assistance, use it with verification, or delegate fully to the AI. Within participants, post-interaction trust strongly predicted the user's delegation choices (OR=14.97, 95% CI [7.14,31.37]). Trust and delegation nevertheless remained distinct. Verification was the dominant policy and often co-occurred with high trust. Delegation also increased with task order after adjustment for momentary and participant-level trust (OR=1.22, 95% CI [1.02,1.45]), indicating an experience-related component that self-reported trust ratings alone could not capture. These findings motivate a shift from intended delegation to a non-intrusive observation of how delegation is enacted in collaborative human-AI tasks. We therefore conclude by proposing enacted delegation traces as a methodological framework for future studies of multimodal and agentic AI use.

0.13

Looks Fine to Me: Understanding Oversight in Agentic Interactions

cs.HC ·2026-10-20 ·Dinithi Dissanayake, Yasith Samaradivakara, Hyung Woon Lee, Yadeesha Weerasinghe, Sanjay Saha +2 more
similarity 0.58
Abstract

Agentic AI systems are deployed to execute complex tasks across domains such as writing, coding, and data analysis. As execution is offloaded to AI agents, the human role shifts toward oversight. Effective oversight requires recognizing when closer scrutiny is needed, even though flawed outputs may appear convincing. Our formative study (N=12) revealed that users accepted plausible outputs without thorough evaluation, and only recognized these lapses in retrospect. To examine this systematically, we conducted a controlled within-subject study (N=36) in which participants completed programming tasks with an agent that produced correct outputs, silent defects, or visible failures. Combining task outcomes, interaction logs, gaze, physiological signals, and self-reports, we examined perceived, behavioral, and effective oversight. Participants were often highly confident while missing silent defects: they performed generic checks, failed to generate diagnostic evidence, or even misinterpreted defect evidence. We discuss how studying oversight across multiple dimensions can support future work on timely, task-calibrated re-engagement.

0.18

Thinking Twice: A Sequential Approach to Nudge Towards Reflective Judgment in GenAI-Assisted Decision Making

cs.HC ·2026-11-06 ·Hüseyin Hussein Keke, Daniel Eisenhardt, Christian Meske
published similarity 0.58
Abstract · Full paper accepted and presented at the 20th International Conference on Wirtschaftsinformatik (WI 2025), Münster, Germany. To appear in th…

Humans increasingly use generative AI (GenAI) for decision-making assistance, yet tend to engage in such situations using quick, intuitive (System 1) rather than deliberate, analytical assessment (System 2) as reflected in Dual Process Theory. Such engagement reduces the effectiveness of human-GenAI interactions and increases associated risks. Our research investigates how to design human-GenAI interactions that promote the transition from System 1 to System 2 thinking in decision-making. Based on the Three-Stage-Model, which suggests humans switch to System 2 when conflicts are detected, we conducted an experiment (n=130) testing sequential human-GenAI interaction as a cognitive forcing nudge during problem-solving tasks. This approach, where humans decide first before receiving AI assistance, enables detection of conflicting decisions. Compared to non-sequential settings (concurrent or no AI assistance), results show the sequential condition significantly enhanced both decision performance and GenAI prompt utilization, providing valuable insights for designing more reflective and thus effective human-GenAI interactions.

0.24

Can People Distinguish Human and AI Agency in Humanoid Teleoperation? A Preliminary Study of Agency Perception

cs.HC cs.RO ·2026-09-06 ·Xiang Li, Koya Dendo, Keigo Minamida, Yuto Nakamura, Per Ola Kristensson +1 more
published similarity 0.58
Abstract · 3 pages, accepted by ACM UIST 2026 as a poster

Can people distinguish between human and AI agency in humanoid teleoperation? To explore this question, we developed \textit{Ghost-in-the-Loop}, a teleoperation framework that supports both human-operated and AI-generated control of a robot's voice, facial expressions, and gestures while maintaining a consistent embodiment. We conducted a preliminary online study ($N=50$) in which participants viewed short interaction clips generated by either a Human Operator or an AI Control and judged the perceived source of control. Results suggest that participants often struggled to distinguish between the two conditions in brief interactions. Qualitative responses indicate that judgments were primarily influenced by perceived naturalness, temporal coordination, and consistency across speech, facial expression, and gesture. These findings provide initial insights into agency perception in embodied human--AI communication and motivate future investigations of blended human--AI telepresence systems.

0.27