CS Paper Radar

Examining the Difference in Human Behavior Between Virtual and Real-World Human-Robot Teaming

Sean Dallas, Absalat Getachew, Motaz AbuHijleh, Andrea Macklem-Zabel, Douglas Zytko, Mark Brudnak, Wing-Yue Geoffrey Louie

cs.RO cs.HC ·submitted 2026-09-16 16:38 UTC · v1
published

Abstract

Prototyping and evaluating human-robot teaming (HRT) scenarios in the real-world is costly. Virtual simulation of HRT scenarios has been adopted as an alternative to conducting user studies in the real-world to investigate user perceptions, behaviors, and performance during human-robot interactions. The consistency of human behavior between the real and virtual-worlds is integral to the validity of utilizing such virtual experimentation. This paper presents a user study examining the difference in human behavior between an HRT scenario conducted in a virtual vs real environment. We employed mixed-methods to examine team performance and human factors during the HRT. Our quantitative results showed a significant difference in workload between the two modalities. Our qualitative analysis expanded on the quantitative results and found differences in participants' strategy, their mental model of robots, and the type of trust they had for robots between modalities.

Comment
8 pages, 2 figures, 1 table, IROS 2026

Closest papers

Nearest neighbours by meaning across the whole index.

MRPilot: Supervising and Intervening LLM-Based Multi-Robot Teams through Mixed Reality

cs.HC cs.RO ·2026-10-05 ·Xiaoran Yang, Xun Qian, Yang Zhan, Nathan Tran, Ziyi Liu +1 more
similarity 0.62
Abstract · 15 pages, 8 figures

Large language models (LLMs) let users direct heterogeneous multi-robot systems (MRS) through natural language, but make task interpretation, robot assignment, and coordination difficult to inspect and change. Based on a formative study with 12 non-expert users, we developed MRPilot, a mixed reality system organized around four stages of supervision and intervention. MRPilot represents robot-team plans and execution states as structured commitments shared across synchronized situated and overview views. Across four stages, it helps users resolve ambiguous references (Forming), review plans before execution (Reviewing), monitor distributed execution (Following), and make robot-level or team-level changes when problems arise (Repairing). In a within-subjects study with 20 participants in a virtual reality-simulated home, MRPilot reduced workload, increased situational awareness, transparency, trust, and perceived control compared with a conventional LLM-based conversational interface using the same LLM planner and robot capabilities. We provide design implications for multi-scale intervention, adaptive supervision, and calibrated reliance in LLM-based MRS.

0.28

A study on human-agent teaming through spoken interaction: the impact of human individual traits and agent characteristics

cs.HC cs.AIcs.DB ·2026-10-02 ·Lara Gauder, Martin Bernardo Meza, Javier Krick, Alejandro Masin, Luciana Benotti +2 more
similarity 0.61
Abstract

Human-agent teams are collaborative systems where humans and agents work interdependently to achieve shared goals. The success of such teams is associated with the human's perception of the agent as a legitimate teammate. This perception is thought to depend not only on the agent's capabilities and reliability but also on social factors. We explore whether simple changes in an agent's communicative behavior can influence the human's perception of the agent's teammate-likeness. To this end, we developed a protocol based on a collaborative game requiring spoken interaction, in which a human subject and a virtual agent collaborate to identify a target object and place it on a board. Each subject interacted with two agents which provided the same task-relevant information but differed in behavior. While one was a neutral tool-like agent, the other was a team-building agent that used simple social strategies such as empathy, politeness, and positivity. Our analysis shows that the team-building agent was perceived as significantly more teammate-like, even in terms of its ability, despite both agents having identical capabilities for game-solving. Further, subjects with a higher propensity to trust automated systems and lower neuroticism tended to have a more favorable perception of the agent. Finally, we found that subjects tended to change their prosodic patterns when talking to the team-building agent, as a classifier based on prosodic features was able to predict the type of agent with better-than-random performance. A practically relevant conclusion is that minimal changes in spoken communication behavior of the agent, easily implemented in a variety of scenarios, can have a significant positive impact on the human's perception of its teammate-likeness.

0.18

Understanding Human-Robot Collaboration through Joint Action

cs.HC cs.RO ·2026-10-27 ·Chengwen Zhang, Jianyang Wang, Chang Liu, Yukun Wang, Yuan Xu +4 more
survey similarity 0.60
Abstract · 26 pages, 5 figures, 32 tables

Human-robot collaboration requires partners to coordinate interdependent contributions toward a shared goal. Joint Action theory provides concepts for understanding this coordination, but how these concepts are instantiated and combined across empirical research remains fragmented. We conducted a systematic review following PRISMA 2020, identifying 83 empirical papers published between 2021 and 2026. Through theory-driven framework analysis, we organized the literature into nine constructs across three capacities: Representation, Prediction, and Integration. We analyze how these constructs are distributed and co-occur across studies, revealing where empirical evidence concentrates and where it is absent. By mapping empirical HRC research onto a unified Joint Action framework, this review offers a coherent overview of HRC studies, identifies gaps in current evidence, and provides guidelines for future research.

0.29

Cognitively Constrained Partners for Zero-Shot Coordination in Real-Time Partitioned Control

cs.LG cs.HC ·2026-10-22 ·Ash Simmons, George De Ath, Richard M. Everson
published similarity 0.59
Abstract · 30 pages, 6 figures, 5 tables. Accepted at the NeurIPS 2026 Workshop on Human-AI Coevolution: Measuring Human-Agent Teams in the Agentic Era

Effective human-machine teaming requires agents that coordinate with unpredictable human partners. Reinforcement learning from live human feedback or large behavioural datasets is expensive, while purely simulated training overfits to artificial conventions and fails at Zero-Shot Coordination with real people. We propose a framework for training teaming agents against Cognitively Constrained Partners: simulated partners whose diversity emerges from procedural constraints grounded in human physiological literature, specifically reaction delays and perceptual limitations, rather than from reward shaping or historical training checkpoints. In a modified Lunar Lander environment, we report preliminary evidence that these partners play more like real people than unconstrained agents do, and that agents trained against them match the strongest cross-play baselines while being the only ones whose performance does not depend on how their training partners were generated. We also describe a human-participant study designed so that its rankings can be compared directly against cross-play, testing how well that standard evaluation predicts real human-agent teaming.

0.25

RealCLI: Toward a Unified Evaluation of Human-Agent Collaboration for CLI Agents

cs.CL ·2026-10-15 ·Yunqi Liu, Peiyu Wang, Bingli Wang, Leo Luo, Xuanwu Song +19 more
benchmark similarity 0.59
Abstract

Command-line interface (CLI) agents increasingly execute longer-horizon tasks, yet existing benchmarks either assume that user instructions remain fixed throughout execution or evaluate task outcomes without capturing runtime collaboration quality. In this paper, we present REALCLI, a paired benchmark that unifies the evaluation of task execution and human-agent collaboration quality on realistic CLI tasks. To comprehensively measure collaboration, REALCLI builds a five-dimensional framework, operationalized into 22 sub-dimensions and 51 interaction scenario templates. REALCLI provides two modes for paired comparison: REALCLI-EXEC supplies complete specifications upfront, whereas REALCLI-COLLAB introduces a scripted user that delivers progressive information, corrections, constraint revisions, and authorization decisions. Evaluations reveal profound capability gaps in frontier agents: the best configuration attains an average score of only 49.2%, and when evaluated by the joint collaboration score ($J_{\mathrm{collab}}$), the peak performance reaches merely 37.6%. These results show that task effectiveness diverges sharply from collaboration quality, with models consistently struggling in steerability, trust calibration, and safety boundary management. REALCLI provides a rigorous, reproducible foundation for measuring and advancing collaborative CLI agents.

0.39

A Scoping Review and Experimental Study on Reinforcement Learning from Human Feedback for Human-Robot Collaboration

cs.HC cs.AIcs.RO ·2026-10-07 ·Alexandra Coroiu, Andrea Vogt, Viktor Werbilo, Andreas Poppele, Johann Christensen +1 more
similarity 0.58
Abstract

Human-Robot Collaboration (HRC) can facilitate mass customisation in Industry 4.0, with Reinforcement Learning from Human Feedback (RLHF) representing a promising approach for developing safe AI-based robots. Practical challenges remain regarding safety during AI development, human feedback quality, and bidirectional human-robot adaptation. We conducted a scoping review of RLHF in HRC systems, mapping methods that address these challenges. Following PRISMA guidelines, we screened 199 records and included 20 peer-reviewed publications (2020-2025) spanning multiple HRC domains. To our knowledge, this is the first review focused on the bidirectional, closed-loop design of RLHF. Our review found multiple feedback modalities enabling data collection in various feedback formats. Collected data can be integrated at different stages of AI training, resulting in a multi-step development process. Pilot experiments are commonly used to evaluate HRC systems based on both human and robot metrics. To empirically test a key gap identified in the review, we conducted a between-subjects VR experiment comparing system- and user-initiated feedback on robot proxemic behaviour for safe navigation. Using Bayesian models, we analysed the relation between the collected feedback and safety metrics: psychological safety (post-experiment questionnaire) and physical safety (inverse time-to-collision). Results show that user-initiated feedback captures perceived safety better than system-initiated feedback, indicating that feedback timing directly affects feedback quality. Our review and experiment findings show that RLHF relies on appropriate feedback methods to ensure AI safety in HRC, and future RLHF research should prioritise realistic HRC experiments evaluating the effects of feedback collection methods on relevant human and robot metrics.

0.13

TeamCAMS: An Open-Source Research Platform for Studying Human Behaviour in Human-AI Teams

cs.HC ·2026-09-17 ·Amos Brocco, Alain Chavaillaz, Andreas Sonderegger, Juergen Sauer
code/data similarity 0.58
Abstract

In this article, we present TeamCAMS (Cabin Air Management System), a collaborative work environment for simulating human-AI (artificial intelligence) interaction for scientific research. The article outlines how several psychological theories guided the development of this multiple-task simulation. Modelling a process control environment, previous versions of TeamCAMS have already been used in empirical studies to address a wide range of research questions (e.g., comparing different forms of automation, evaluating impact of automation reliability, effects of stress on multiple-task performance). Outlining the technical possibilities offered by TeamCAMS, the article points out how its latest version offers researchers the possibility of addressing a set of new research questions including problems associated with teamwork (e.g., within-team conflict, distributed teamwork) and human-AI interaction. Finally, we will outline how this simulation environment can be enhanced further still to address research questions in new fields (e.g., automation of leadership). To promote transparency, reproducibility, and further development, TeamCAMS is made available to the research community under an open-source license.

0.25

Synchrony, Affect, and Perceived Autonomy in Virtual Robot-Facilitated Anxiety Support Groups

cs.HC ·2026-11-16 ·A'di Dust, Mohammad Soleymani, Mina Kian, Maja Matarić
similarity 0.57
Abstract · 17 pages, 7 figures

Socially assistive robots could expand access to group-based mental health support when skilled human facilitators are unavailable. However, their effectiveness will depend on whether they foster productive interactions among participants. We examined dyadic head-movement synchrony among 96 participants across 18 virtual robot-facilitated anxiety support group sessions. A human-controlled virtual robot, via a Wizard-of-Oz (WoZ) system, facilitated all sessions. We found that stronger peak synchrony predicted lower post-session negative affect. In contrast, speakers who more often followed listeners' movements reported higher negative affect and state anxiety, demonstrating that synchrony strength and direction capture distinct interpersonal processes. Participants who believed the robot was autonomous showed more genuine synchrony and stronger peaks than those who believed it was human-operated; more negative attitudes toward robots also predicted greater synchrony. These findings support synchrony as an objective, behaviorally grounded measure for evaluating robot-facilitated groups and a potential signal for guiding future autonomous facilitators.

0.19