CS Paper Radar

Design and Evaluation of a Collaborative Teleoperation Interface for Multi-Operator Multi-Robot Systems in Challenging Environments

Alperen Kenan, Juan José García Cárdenas, Adriana Tapus, Paul Bremner, Manuel Giuliani

cs.RO cs.HC ·submitted 2026-10-10 15:32 UTC · v1
published

Abstract

Multi-robot systems that combine the complementary capabilities of specialised platforms are critical for complex missions in challenging, inaccessible environments. Current deployments assign an operator or operator team to each robot, with collaboration relying on operator-to-operator communication to manually coordinate heterogeneous platforms with incompatible interfaces. This operator-dependent approach increases cognitive load and limits scalability across robots from different manufacturers. This study designs and evaluates a shared-visualisation collaborative teleoperation interface to improve operator coordination, situational awareness, and the task performance in multi-operator, multi-robot systems. A comparative user study with 16 participants compared three teleoperation conditions: co-located operators with a shared interface, remote operators with a shared interface, and co-located operators with individual interfaces. Participants controlled two simulated robots in a nuclear power plant digital twin to complete disaster response tasks. Performance metrics, physiological data, and behavioural measures were recorded to assess task performance, cognitive load, and operator awareness across conditions. The co-located shared-interface condition achieved the fastest mission completion (292.8 s), outperforming remote operation (351.6 s) and individual interfaces (385.1 s), with the fewest collisions (0.9 per trial) and highest situational awareness accuracy (91.1 %). Physical co-presence was associated with reduced cognitive load, with NASA-TLX scores increasing from 38.2 in co-located to 57.7 in remote operation, supported by consistent physiological indicators. These results demonstrate that co-presence and shared interfaces provide complementary benefits, supporting workload regulation and mutual awareness respectively in multi-operator, multi-robot teleoperation.

Comment
9 pages, 8 figures, accepted for presentation at the 2026 IEEE Conference on Telepresence in Bristol, United Kingdom, November 9-13, 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Affective Shared Autonomy: Temporal Affect Dynamics and Subjective Evaluation in Bimanual Teleoperation Tasks

cs.RO cs.HC ·2026-09-17 ·Zhengji Liang, Guiyin Tian, Sijin Qu, Hainan Liu, Shiyan Hu
similarity 0.67
Abstract

Physical teleoperation integrates human cognitive flexibility with robotic precision, yet demanding manipulation tasks frequently induce severe cognitive workload, acute frustration, and execution breakdown. Conventional shared autonomy paradigms rely primarily on task-based rules, such as spatial error boundaries, which disregard the operator's transient affective state and risk misaligned control interventions. To address this limitation, we propose an affect-aware shared autonomy teleoperation framework that dynamically modulates robotic assistance based on real-time operator state estimation. The system estimates operator affective states from synchronized facial video, cardiac signals, and bilateral arm kinematics, outputting a seven-state affective distribution and a three-category operational abstraction (neutral, productive, adverse). Affect-aware assistance is selectively triggered when the user is detected in a continuous adverse state, preserving task-positive engagement without unnecessary disruption. The empirical user study ($N = 30$) confirms that the proposed affective assistance increases the productive states by up to 39.7% without compromising user agency. The collected dataset represents the first multimodal dataset that provides continuous visual, physiological, and operator's bilateral motion tracking of temporal affective state shifts during bimanual teleoperation. Our multimodal fusion model outperforms zero-shot baselines (Qwen, MiniCPM-V) in tracking temporal state dynamics. This real-world deployment offers a new human-centric framework that integrates visual, physiological, and motion tracking for physical human-robot interaction.

0.27

GHOST in the Robots: Real-Time Exocentric Dual-Robot VR Teleoperation from Onboard Cameras

cs.RO ·2026-08-29 ·Yichen Wei, Faisal Zaghloul, Soujanya C Aryal, Aanya K. Agrawal, Chengfan Li +3 more
top venuecode/data similarity 0.64
Abstract

Teleoperating multiple robots simultaneously enables additional views and coordinated control. Yet, it poses fundamental challenges: the system must present sensor data cohesively and allow operators to manage multiple robot bases, arms, and cameras while maintaining low latency. Current multi-robot teleoperation systems require multiple operators, rely on autonomy, or restrict operators to high-level commands. We present GHOST: an open-source VR teleoperation system that enables single operator control of two mobile manipulators via direct lowlevel commands using only onboard sensing. GHOST creates an exocentric 3D workspace by aligning real-time point clouds from the robots' RGB-D cameras, where scene coverage is improved through learning-based completion to aid operator spatial awareness. For control, the operator uses a mode-switching architecture to command either robot individually or both robots simultaneously. Experiments with 15 novice participants demonstrate 1.6-4x the success rate of an off-the-shelf tablet interface. For experts across nine challenging dual-robot tasks, our system enabled completion of two tasks that were infeasible with the tablet, and was 1.47x faster on average than the tablet. Website and code: https://h2r.github.io/GHOST/.

0.38

Evaluating Lightweight Contact Feedback in Dual-Operator Collaborative Humanoid Teleoperation

cs.RO cs.HC ·2026-10-16 ·Austin Graves, Chuizheng Kong, Yunho Cho, Yuanshao Yang, Advaith Nidumukkala +2 more
similarity 0.63
Abstract · 8 pages, 4 figures, 2 tables. Project website: https://xrt-teleop.github.io

Dual-operator humanoid teleoperation enables collaborative manipulation through two separate robots, but operators lack direct awareness of robot-object contact that is normally available during physical interaction. To address this, we present eXtended Reality Teleoperation (XRT), an extended reality teleoperation framework that returns a robot's fingertip contact through lightweight vibrotactile and/or auditory feedback, and a synchronized dual-humanoid testbed built on it for two operators. Sixteen dyads performed a collaborative crate lift-and-flip under vision only and the three feedback conditions in a counterbalanced within-subject design with verified feedback-channel delivery. Three results follow. First, operators rated our finger-localized vibrotactile display as more effective than our open-ear sonification: all 32 ranked it above the sonification, 18 ranked it most effective, and its cues were rated far easier to perceive and interpret than the auditory cues. Second, completion time, failures, workload, discomfort, and partner awareness showed no detected condition differences, with wide completion-time intervals and smaller meaningful effects unresolved. Third, performance changed substantially with practice, especially on the harder flip-over task: completion time improved across blocks, and attempt-to-attempt variation within a dyad (SD x1.51) was larger than the largest condition contrast (x1.16). Exploratory analyses detected no difference in immediate grip or motion responses between feedback conditions. For novice collaborative teleoperation, our fingertip vibrotactile cues were rated more effective and easier to interpret than our auditory cues, while their effect on task performance remains unresolved.

0.15

A Taxonomy on Collective Awareness

cs.RO ·2026-10-05 ·Guillermo GP-Lenza, Miguel Fernandez-Cortizas, Martín Molina, Ricardo Sanz, Pascual Campoy
position similarity 0.62
Abstract

As robotic teams tackle increasingly complex tasks in dynamic and unstructured environments, effective coordination requires agents to maintain accurate, aligned representations of their environment, teammates, and mission state. We argue that Mutual Awareness, Shared Situational Awareness, and Team Situational Awareness --- three concepts widely invoked in the multi-robot systems literature --- are not interchangeable: they form a containment hierarchy in which each type subsumes the previous in scope, and span a distribution spectrum from fully individualized understanding (MA) to fully uniform understanding (SSA), with TSA occupying a mixed position. We establish this through a two-dimensional taxonomy organized along the object of awareness and awareness distribution axes. Treating these terms as synonyms obscures the precise coordination requirements each imposes on a robotic system. A Search and Rescue case study with heterogeneous aerial and ground robots grounds each taxonomy position in concrete coordination requirements. These findings provide a conceptual foundation for principled specification, design, and comparison of collective awareness in multi-robot systems.

0.24

Location-Independent Robot-Assisted Finishing Using Digital Twins and Extended Reality

cs.HC ·2026-09-08 ·Jose Outeiro, Jia Holt, Tero Kaarlela, Khalil Chakal
similarity 0.62
Abstract · 6 pages, 5 figures, 2027 IEEE/SICE International Symposium on System Integration

This paper presents a cyber-physical system (CPS) for location-independent programming, supervision, training, and teleoperation of a Robot-Assisted Finishing (RAF) system used to post-process metal additive-manufactured (AM) components. A digital twin (DT) built in Unity is delivered to the operator as a WebGL application that supports both desktop and immersive modes through WebXR-compatible devices. Moreover, it exchanges robot state and pose commands with a collaborative robot through a Message Queuing Telemetry Transport (MQTT) broker. The DT enforces kinematic and collision constraints before a pose is released to the physical robot, and augments the virtual component with a color map of the surface topography that supports operator decisions on part repositioning or process termination. The architecture was validated on a specially designed physical RAF system. A steady-state joint synchronization error of 0.12 deg and a mean round-trip latency of 563 ms were measured, which is adequate for supervisory programming and intermittent teleoperation.

0.20

Co^2Gen: Collaboratively Generating Task Plans for Decentralized Multi-Robot and Human Collaboration

cs.RO ·2026-10-22 ·Yi Loo, Siddhant Shrivastava, Minh Long Pham, Malika Meghjani
similarity 0.61
Abstract

Visual Language Action (VLA) models are designed to be generalist robot models that can be implemented across embodiments, tasks and environments. However, their ability to collaborate with other robots and humans remains underexplored. In this work, we propose a framework to automate the generation of multi-robot collaborative data for decentralized VLA training without requiring human teleoperation. Our framework first utilizes a decentralized collaborative planning approach to generate multi-robot collaborative robot plans in the form of an executable function. We then utilize the collaborative plan to produce collaborative demonstration data to finetune VLA models in a decentralized manner. We show that our approach produces demonstration data that when used to finetune VLA models on decentralized collaboration tasks matches the task performance of VLA models finetuned on human-teleoperated collaborative data.

0.19

Remote Surfaces at Your Fingertips: Electrovibration-Based Tactile Feedback for Robot Teleoperation via Touchscreen Interfaces

cs.RO cs.HC ·2026-08-22 ·Alperen Kenan, Juan José García Cárdenas, Adriana Tapus, Paul Bremner, Manuel Giuliani
published similarity 0.61
Abstract · 8 pages, 10 figures, accepted for presentation at the 2026 IEEE Conference on Telepresence in Bristol, United Kingdom, November 9-13, 2026

Enabling operators to perceive and interact with remote environments naturally is a fundamental challenge in robotic teleoperation. This is especially critical in tasks involving physical interaction, where real-time haptic awareness improves operational safety and effectiveness. Existing kinesthetic haptic feedback methods suffer from instability during rigid surface contacts and remain sensitive to communication delays, while visual cue-based force feedback imposes additional cognitive load and limits sustained situational awareness. This work presents a teleoperation interface that conveys remote surface interactions to the operator through electrovibration-based tactile feedback, enabling naturally mapped force reflection while avoiding the stability issues associated with kinesthetic feedback and the latency limitations of mechanical actuators. A user study (N=21) evaluated interface usability, sense of presence, and operator workload under two force reflection conditions: visual feedback and electrovibration-based tactile feedback. Characterisation experiments further assessed path-following accuracy and response time across both conditions. Results show that tactile feedback significantly reduced response time by 15.35% (p=0.002, d=0.96) and increased the sense of presence by 31% (p<0.001, d=0.90) compared to visual feedback, while imposing comparable workload and usability across both conditions. These findings demonstrate that electrovibration-based tactile feedback is a viable and effective modality for robot teleoperation, improving operator responsiveness and sense of presence in contact-rich manipulation tasks, with direct applicability to safety-critical domains such as nuclear maintenance.

0.20

The Missing Touch: Spatially Distributed Tactile Feedback Brings Teleoperation Closer to Human Dexterity

cs.RO ·2026-08-19 ·Rohan Kota, Gregory Reardon, J. Edward Colgate
similarity 0.61
Abstract

A fundamental challenge in robotic teleoperation is enabling an operator to control a remote robot as effortlessly and intuitively as their own hands. Despite the growing use of teleoperation to collect demonstration data for training autonomous robot policies, teleoperated robot performance still falls significantly short of human dexterity, even for basic tasks. Here, we present evidence that a key factor contributing to this performance gap is the absence of spatially distributed tactile feedback. Using a two-degree-of-freedom (DoF) bilateral force-feedback telemanipulator paired with a 32-DoF tactile fingertip display, we show that operator performance improves significantly when localized deformations on the remote manipulator are faithfully reproduced on the operator's fingertip. In a series of teleoperation tasks, reproducing distributed contact information not only accelerated task performance but also brought teleoperated movements closer to natural human behavior by minimizing corrective actions and task completion steps, thereby reducing the deviation between teleoperated and natural trajectories by 29$\unicode{x2013}$79%. Furthermore, we found that increasing the resolution of the tactile feedback$\unicode{x2014}$by refining how finely the measured displacements were quantized for reproduction$\unicode{x2014}$compressed the state-space distribution of teleoperated motions, which has been associated with improved training outcomes for autonomous robot policies. Together, these results suggest that spatially distributed tactile feedback is essential for closing the gap between human and teleoperated dexterity and training the next generation of autonomous robots.

0.30