CS Paper Radar

Better Situational Awareness in AR-HRC? A Comparative Study of Augmented Reality and Mobile Interfaces for Human-Robot Collaboration

Zhehan Qu, Christian Fronk, Jaewoong Jeong, Pavel Manakhov, Maria Gorlatova

cs.HC ·submitted 2026-09-01 16:01 UTC · v2

Abstract

Augmented reality (AR) facilitates human-robot collaboration (HRC) by enabling in-situ spatial visualizations of the robot and the joint task. However, in safety-critical HRC scenarios such as search-and-rescue, spatial visualizations may also reshape visual attention in ways that create competing situational awareness (SA) demands, potentially introducing new safety concerns. While prior AR-HRC work suggests potential benefits for SA, rigorous evaluations that jointly consider robot and environmental awareness across multiple levels of SA remain limited. We address this through a between-subjects study with 30 participants comparing custom AR and mobile interfaces presenting equivalent information, measuring robot and environmental SA with the Situation Awareness Global Assessment Technique (SAGAT) across all three levels, with concurrent eye tracking to identify the attentional mechanisms underlying any SA differences. Both interfaces achieved high usability; relative to the mobile baseline, AR improved perception-level awareness of the robot but yielded no gains in higher-level robot awareness or in environmental awareness at any level. Gaze analysis explained this: AR freed attention from the map, but that attention was re-invested in the conformal visuals rather than the physical environment. Freeing the eyes from a screen is not the same as directing them to the world, a distinction AR interfaces for safety-critical HRC must design around.

Comment
11 pages, 9 figures, 1 table. Zhehan Qu and Christian Fronk contributed equally

Closest papers

Nearest neighbours by meaning across the whole index.

NavSight in the Wild: Understanding Real-World Use of a Mobile Augmented Reality Application for People with Low Vision in Outdoor Navigation

cs.HC ·2026-08-13 ·Yuheng Wu, Kexin Zhang, Ben Kosa, Ru Wang, Sanbrita Mondal +1 more
published similarity 0.64
Abstract · 32 pages, 11 figures

The ability to navigate outdoors safely and independently is crucial yet challenging for people with low vision (PLV). While various augmented reality (AR) systems for low vision have been designed and evaluated in ideal lab environments, no research has investigated their real-world feasibility and challenges. We present NavSight, a mobile AR application that assists PLV in outdoor navigation by recognizing important outdoor objects (e.g., curb, vehicle) and rendering real-time visual augmentations. Through a seven-day diary study with 12 PLV in real-world settings, we characterize the impact of NavSight on scene perception, users' configuration strategies on what objects to augment and how to augment them across scenarios, how users made sense of and responded to recognition errors, and the social acceptability of using NavSight in public. We further identify environmental factors affecting recognition, such as weather conditions, lighting and shadows, and nonstandard road markings and textures, as well as usability issues in daily use. We discuss these real-world challenges and derive design implications for future AI-powered assistive AR systems for outdoor use.

0.25

Beyond Activation: Gaze Invocation with Visible Status for an Embodied AR Assistant in Co-Located Collaboration

cs.HC ·2026-10-07 ·Chenrui Ma, Yoshio Ishiguro, Qing Zhang
similarity 0.63
Abstract · 11 pages, 9 figures, 2 tables

Wake words face an inherent trade-off: higher sensitivity reduces missed commands but increases accidental activations. In co-located augmented reality (AR), the system must also determine whether the user is speaking to the assistant or to a nearby person. We introduce a new perspective on this problem: combining gaze-based address with an embodied assistant, visible listening status, and turn management across activation, continued interaction, and release. In our implementation, users look at an assistant anchored in the scene and see activation progress and listening status on its body. Sustained gaze opens a local interaction channel; speech and playback keep it open when attention returns to the task; and inactivity closes it. We evaluated this design with 25 participants. After selecting the assistant's placement and dwell duration, each participant worked with a partner to plan a trip while using the assistant. The study recorded 500 interaction outcomes, including 496 assistant-directed utterance attempts. Gaze acquisition completed before speech for 490 of these attempts (98.8\%): 467 began while the channel remained open, whereas 23 began after it had been released. The other 6 attempts began before acquisition completed. Among 102 reviewed attempts in which gaze left after acquisition but before speech, the configured retention policy kept the channel open for 79 and released it before 23. The results show that an embodied target with visible status can support clear entry into an assistant interaction while revealing a different problem at release: after users look back to their work, they may not see that the assistant has stopped listening. We contribute the implemented gaze-invocation design and design implications for communicating assistant state after visual attention moves elsewhere.

0.29

Evaluating the Impact of Adaptive Extended Reality on Human-Robot Interaction Across the Reality-Virtuality Continuum

cs.RO cs.HC ·2026-09-25 ·Carl Tornberg, Alicia Torck, Lotfi El Hafi, Tadahiro Taniguchi
code/data similarity 0.61
Abstract · Submitted to Advanced Robotics, Special Issue on "Next Generation Cognitive Robotics: Nurturing Embodied Intelligence for a Symbiotic Future…

As populations in developed countries age and labor shortages intensify, Cybernetic Avatars (CAs) are proposed to extend human capabilities through robotic embodiments, requiring effective Human-Robot Interaction (HRI) frameworks. Extended Reality (XR), an umbrella term for Augmented Reality (AR), Augmented Virtuality (AV), and Virtual Reality (VR), offers such interfaces, but prior research typically fixes the XR modality without evaluating its effect on task outcomes. This study examines whether the XR modality impacts HRI performance and whether an adaptive interface adjusting the level of virtuality along the Reality-Virtuality Continuum (RVC) at runtime improves it. A custom XR application interfaced with a mobile manipulator supports immersive control and runtime modality switching. In a within-participant multi-room pick-and-place experiment comparing fixed AR, AV, and VR with dynamic RVC through task metrics, the NASA-TLX, and the System Usability Scale (SUS), this study demonstrates that 1) the fixed reality modality affects HRI results, and 2) dynamically changing the modality along the RVC improves them. AR yielded significantly lower mental demand, effort, and frustration than AV and VR, while the dynamic RVC condition achieved the highest throughput and lowest workload, highlighting the value of adaptive XR interfaces for human-robot symbiosis. The implementation is available at https://github.com/CarlTornberg/XR-HRI.

0.38

Human Situational Awareness and Workload during Heterogeneous Autonomous Vehicle Hazard Monitoring: A Cross-Platform Study

cs.HC ·2026-10-12 ·Ziyue Wang, Yang Xing, Argyrios Zolotas, Adolfo Perrusquia, Weisi Guo +1 more
similarity 0.61
Abstract

As autonomous ground and aerial vehicles become increasingly capable, human operators are transitioning from direct control to supervisory monitoring roles. Maintaining Situational Awareness (SA) during rare but safety critical hazard events remains a fundamental challenge, particularly when operators must monitor heterogeneous autonomous platforms with different spatial and cognitive demands. This study develops a unified experimental framework for investigating human SA and workload during supervisory monitoring of autonomous ground and aerial vehicles under hazard conditions. Eighteen participants completed six within-subject conditions combining three interface modalities, namely, visual-only, haptic-only, and visual-haptic, with two platform types, ground vehicle and aerial drone. SA was measured using Situation Awareness Rating Technique (SART), while workload was assessed using the NASA Task Load Index (NASA-TLX). The key findings are: 1) Aerial monitoring produced significantly lower overall SA, higher situational demand, and greater workload than ground monitoring, driven by elevated Demand and reduced Understanding rather than diminished Supply. 2) Workload was consistently coupled with situational demand across all conditions, suggesting a stable SA-workload mechanism. 3) Both haptic-only and visual-haptic significantly improved overall SA relative to visual-only without increasing workload, demonstrating that interface design can enhance awareness within the existing workload envelope. These findings establish a cross-platform behavioural baseline for designing multimodal interfaces that enhance operator SA without elevating workload in heterogeneous autonomous vehicle systems.

0.27

A Taxonomy on Collective Awareness

cs.RO ·2026-10-05 ·Guillermo GP-Lenza, Miguel Fernandez-Cortizas, Martín Molina, Ricardo Sanz, Pascual Campoy
position similarity 0.61
Abstract

As robotic teams tackle increasingly complex tasks in dynamic and unstructured environments, effective coordination requires agents to maintain accurate, aligned representations of their environment, teammates, and mission state. We argue that Mutual Awareness, Shared Situational Awareness, and Team Situational Awareness --- three concepts widely invoked in the multi-robot systems literature --- are not interchangeable: they form a containment hierarchy in which each type subsumes the previous in scope, and span a distribution spectrum from fully individualized understanding (MA) to fully uniform understanding (SSA), with TSA occupying a mixed position. We establish this through a two-dimensional taxonomy organized along the object of awareness and awareness distribution axes. Treating these terms as synonyms obscures the precise coordination requirements each imposes on a robotic system. A Search and Rescue case study with heterogeneous aerial and ground robots grounds each taxonomy position in concrete coordination requirements. These findings provide a conceptual foundation for principled specification, design, and comparison of collective awareness in multi-robot systems.

0.24

Exploring Object Recall in an Object-Rich AR Office Scene

cs.HC ·2026-10-13 ·Sangita Kunapuli, Shane Dirksen, Radha Kumaran, Yanxiu Jin, You-Jin Kim +2 more
published similarity 0.61
Abstract · 11 pages, 9 figures. Accepted to IEEE ISMAR 2026 and published in IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 32, …

Augmented reality (AR) changes how users perceive and attend to their surroundings. Prior work suggests AR can reduce attention to the physical world, but it remains unclear whether this holds in environments dense with both virtual and physical objects, and which factors most affect object recall. This study examines recall in a mixed-reality office scene after an object-selection task, testing object-level factors (virtuality, size), context-level factors (semantic congruency, saliency), and user-level factors (cueing, dual tasking). Cueing informed participants of a later recall test, while dual tasking added cognitive load through a secondary audio task. Virtual objects were recalled more accurately than physical ones. Cueing produced a modest improvement in recall of present objects. Using playback software to reconstruct each participant's view throughout the trials, we computed object-level saliency and found a weak positive correlation with recall, slightly stronger than the effects of virtuality, but not reaching statistical significance. These findings have implications for AR interface design and for building mixed-reality environments that better support cognition during real-world tasks.

0.30

How Contextual Information in Physical Environments Shapes Decisions with AI

cs.HC ·2026-11-02 ·Omar Elgohary, Amanda Elvarsdottir, Vannessa Lee, Zhu-Tian Chen
similarity 0.61
Abstract

AI assistants in augmented reality may advise users about parts of the physical world they cannot directly observe. We ask whether contextual information from the situation helps users make better decisions with fallible AI. In an AR visual- search study (N = 32), we varied target prevalence and scene visibility while participants judged whether a key was present, with or without AI assistance. Lowering target prevalence and revealing part of the desk together improved observed AI-assisted accuracy, but our analysis did not demonstrate that this gain came from participants making better use of context. Context also changed which answers seemed plausible: when keys were rare, participants became more skeptical of Present suggestions, but this shift applied to both correct and incorrect suggestions rather than improving selective acceptance of correct advice. Revealing part of the desk also increased decision time, with little evidence of better AI use. Overall, contextual information can shape AI- assisted decisions and increase decision effort without necessarily helping users judge when the AI is correct.

0.13

Gaze Prompts: Temporally Dense Human Attention for Vision-Language-Action Fine-Tuning

cs.RO ·2026-09-28 ·Yihan Zhou, Rui Yan, Mingcong Li, Zheyuan Huang, Xu Yang +2 more
dataset similarity 0.60
Abstract

Vision-Language-Action (VLA) fine-tuning pairs images with actions at every step, yet typically provides only a task-level language instruction, leaving moment-to-moment visual relevance implicit. We introduce \emph{eye-tracker-supervised gaze prompting}, which uses gaze recorded during VR teleoperation to provide frame-level visual guidance for VLA fine-tuning. During training, recorded gaze locations are rendered as crosshairs on the robot's head-camera images. At deployment, a lightweight predictor estimates gaze locations from recent images and the instruction, supplying the same type of visual prompt without an eye tracker or changes to the policy architecture. Instantiated with $π_0$, gaze prompting increases mean success from $26.3\%$ to $56.0\%$ across six real-world bimanual manipulation tasks, with gains also observed when a single policy is trained on all six tasks. We release \textsc{GazeMani}, a dataset of $1{,}200$ teleoperated trajectories with synchronized gaze.

0.20