CS Paper Radar

Cognitive and Behavioural Effects of Haptic Feedback for ROV Teleoperation in Hazardous Environments

Joshua Brown, Ildar Farkhatdinov, Michael Jenkin

cs.RO cs.HCeess.SY ·submitted 2026-11-04 15:55 UTC · v1
published

Abstract

Teleoperating a Remotely Operated Vehicle (ROV) in complex underwater environments presents unique challenges. Dynamic, three dimensional obstacles and unseen water currents pose real risks to expensive robotic hardware in environments in which recovery can be next to impossible. This paper explores the cognitive effects of haptic feedback to improve vehicle safety in such environments. An in-water study is analysed with a ROV exposed to controlled cross-currents, with environmental information presented via a soft and vibrating haptic touchpad. Results demonstrate that while encountering an unseen cross-current significantly increases an operator's physical and temporal demand, the introduction of haptic feedback mitigates these factors and improves self-assessed performance. Interestingly, the haptic condition also produced the highest levels of mental demand and operator frustration. Despite this increased mental workload, operators drove the ROV significantly faster when receiving haptic cues compared to the current-only condition, indicating a higher level of user confidence. Ultimately, these findings suggest that while haptic feedback may initially require more cognitive processing to interpret unseen hazards, it successfully enhances operator awareness and promotes more confident navigation in hazardous environments.

Comment
Accepted for IEEE Telepresence 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Remote Surfaces at Your Fingertips: Electrovibration-Based Tactile Feedback for Robot Teleoperation via Touchscreen Interfaces

cs.RO cs.HC ·2026-08-22 ·Alperen Kenan, Juan José García Cárdenas, Adriana Tapus, Paul Bremner, Manuel Giuliani
published similarity 0.70
Abstract · 8 pages, 10 figures, accepted for presentation at the 2026 IEEE Conference on Telepresence in Bristol, United Kingdom, November 9-13, 2026

Enabling operators to perceive and interact with remote environments naturally is a fundamental challenge in robotic teleoperation. This is especially critical in tasks involving physical interaction, where real-time haptic awareness improves operational safety and effectiveness. Existing kinesthetic haptic feedback methods suffer from instability during rigid surface contacts and remain sensitive to communication delays, while visual cue-based force feedback imposes additional cognitive load and limits sustained situational awareness. This work presents a teleoperation interface that conveys remote surface interactions to the operator through electrovibration-based tactile feedback, enabling naturally mapped force reflection while avoiding the stability issues associated with kinesthetic feedback and the latency limitations of mechanical actuators. A user study (N=21) evaluated interface usability, sense of presence, and operator workload under two force reflection conditions: visual feedback and electrovibration-based tactile feedback. Characterisation experiments further assessed path-following accuracy and response time across both conditions. Results show that tactile feedback significantly reduced response time by 15.35% (p=0.002, d=0.96) and increased the sense of presence by 31% (p<0.001, d=0.90) compared to visual feedback, while imposing comparable workload and usability across both conditions. These findings demonstrate that electrovibration-based tactile feedback is a viable and effective modality for robot teleoperation, improving operator responsiveness and sense of presence in contact-rich manipulation tasks, with direct applicability to safety-critical domains such as nuclear maintenance.

0.20

Human Situational Awareness and Workload during Heterogeneous Autonomous Vehicle Hazard Monitoring: A Cross-Platform Study

cs.HC ·2026-10-12 ·Ziyue Wang, Yang Xing, Argyrios Zolotas, Adolfo Perrusquia, Weisi Guo +1 more
similarity 0.64
Abstract

As autonomous ground and aerial vehicles become increasingly capable, human operators are transitioning from direct control to supervisory monitoring roles. Maintaining Situational Awareness (SA) during rare but safety critical hazard events remains a fundamental challenge, particularly when operators must monitor heterogeneous autonomous platforms with different spatial and cognitive demands. This study develops a unified experimental framework for investigating human SA and workload during supervisory monitoring of autonomous ground and aerial vehicles under hazard conditions. Eighteen participants completed six within-subject conditions combining three interface modalities, namely, visual-only, haptic-only, and visual-haptic, with two platform types, ground vehicle and aerial drone. SA was measured using Situation Awareness Rating Technique (SART), while workload was assessed using the NASA Task Load Index (NASA-TLX). The key findings are: 1) Aerial monitoring produced significantly lower overall SA, higher situational demand, and greater workload than ground monitoring, driven by elevated Demand and reduced Understanding rather than diminished Supply. 2) Workload was consistently coupled with situational demand across all conditions, suggesting a stable SA-workload mechanism. 3) Both haptic-only and visual-haptic significantly improved overall SA relative to visual-only without increasing workload, demonstrating that interface design can enhance awareness within the existing workload envelope. These findings establish a cross-platform behavioural baseline for designing multimodal interfaces that enhance operator SA without elevating workload in heterogeneous autonomous vehicle systems.

0.27

AQUAACT: A Joystick-Level Underwater Teleoperation Dataset and Closed-Loop Benchmark

cs.RO ·2026-10-18 ·Zhenqi Wu, Trung Tien Dong, Aditya Penumarti, Mohamed A. Fouad, Menna A. Fouad +3 more
benchmark similarity 0.64
Abstract

Underwater robots support a growing range of work, from inspecting offshore structures and pipelines to surveying reefs and accompanying divers. Most of these tasks, however, are still flown by hand by a pilot, which raises the question of whether they can be carried out autonomously instead. On land, vision-language-action (VLA) models already let robots carry out tasks autonomously from language instructions, which makes them a natural candidate for taking the pilot's place. Underwater, however, there is little data to train such models. We take inspiration from how these tasks are flown today: the pilot watches the camera image and steers with a joystick, whose commands the flight controller turns into motion. We therefore record the pilots and introduce AQUAACT, to our knowledge the first publicly available language-annotated underwater dataset pairing camera images with human joystick commands. It comprises 1.9 hours of real-world and 17.9 hours of simulated demonstrations, both flown by human pilots, together with the vehicle's inertial measurements. Experiments with AQUAACT show that its simulated demonstrations transfer to real data and greatly reduce the real demonstrations a policy needs. A closed-loop benchmark of seven policies in simulation, from imitation-learning baselines to VLA models, each trained with its default recipe, shows that these tasks remain far from solved: no policy completes more than a fifth of the episodes. Pretraining on land data gives large VLA models no advantage over small policies trained from scratch, a gap that land-trained models have yet to close underwater.

0.07

Safety-Critical Bilateral Teleoperation for Omnidirectional Aerial Manipulation Using Force-Sensorless Haptic Feedback

cs.RO ·2026-08-22 ·Yubin Kim, Jinwoo Lee, Yongjun You, H. Jin Kim, Jeonghyun Byun
published similarity 0.62
Abstract · 8 pages, 10 figures. Accepted to the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026)

This paper presents a safety-critical bilateral teleoperation framework for omnidirectional aerial manipulators that integrates visual and force-sensorless haptic wrench feedback. Unlike existing approaches that either rely on onboard force/torque sensors or use model-dependent wrench estimates, which may become unreliable under model uncertainties or induce unintended feedback during free-flight, our method implements a hierarchical safety filter based on control barrier functions to avoid such limitations. The safety filter, being the key contribution, explicitly accounts for tracking errors arising from physical interaction between the aerial manipulator and its surroundings while enforcing thrust limits, a factor overlooked despite its critical importance for flight safety. This safety filter adjusts the command from the operator to ensure safe and stable aerial manipulation and avoid motor saturation. The adjustment made by the filter is mapped to haptic feedback, which is intuitive to the operator and conveys information on physical interaction and impending motor saturation. By actual experiments with a hexarotor-based omnidirectional aerial manipulator, we demonstrate that the proposed method avoids haptic feedback during free-flight, provides directionally consistent feedback under physical interaction, and can be operated for diverse manipulative tasks. Moreover, an ablation study further shows that the saturation filter improves interaction stability by explicitly preventing motor saturation and informing the operator of corrective actions.

0.37

Tactile Search: Enhancing Targeting in 3D Space

cs.HC ·2026-09-05 ·Amber Maimon, Iddo Yehoshua Wald, Jonas Keppel, Eunhee Chang, Yoshifumi Kitamura +3 more
top venue similarity 0.61
Abstract · Accepted to the 2026 IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2026)

Visual search is crucial in daily life, from scanning for relevant information to spotting signs of danger. When sensory channels are overloaded or degraded, cognitive tasks can be supported by crossmodal information representations through vibrotactile cues. We introduce Tactile Search, an approach that uses modulation of frequency and amplitude of vibrations to the hands, for guiding attention to the location of objects in 3D space. We evaluated this approach in a competitive VR game where participants searched for targets using both vision and touch. Across two studies -- an in-the-wild demonstration (n=55) and a controlled laboratory experiment (n=28) -- we found that vibrotactile feedback significantly improved performance and increased user confidence. In the combined haptic condition, performance did not differ across target heights. We further analyzed participants' subjective experiences and search strategies highlighting the benefits of the tactile cues. Our findings suggest that Tactile Search can enhance interaction and provide design considerations for integrating haptic search into interactive systems.

0.24

MR-GLi: Mixed Reality-Based Gripper-Linked Overlays for Underwater Robot Arm Teleoperation via Bilateral Control

cs.RO ·2026-09-11 ·Masashi Sasago, Masato Kobayashi, Yuki Uranishi
similarity 0.61
Abstract

Visual torque feedback supports underwater bilateral teleoperation, but the benefit of mixed reality (MR) over conventional monitor presentation remains unclear. We present MR-GLi, an MR interface that spatially registers a reaction torque indicator and wrist-camera image to the robot gripper. Twenty participants performed lift and pick-and-place tasks with rigid and compliant objects in a counterbalanced within-subject comparison with a 2D monitor, using identical visual-feedback content and four-channel bilateral control. MR-GLi provided gripper-linked access to visual feedback while maintaining a similar level of torque-regulation performance to the 2D monitor. Subjective evaluation further indicated reduced perceived burden associated with shifting attention between the workspace and visual feedback. These results demonstrate the feasibility of gripper-linked MR overlays for underwater bilateral teleoperation and highlight the importance of considering information access in addition to task performance. Additional material: https://mertcookimg.github.io/mr-gli/

0.29

ViHaTeleop: A Low-Cost, Lightweight Visual-Haptic Teleoperation System for Dexterous Manipulation Learning

cs.RO ·2026-08-17 ·Fucai Zhu, Yanhou Lai, Paul Maestre, Koichi Hashimoto
published similarity 0.60
Abstract · Accepted to the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 8 pages

Learning from demonstration is a promising approach for dexterous manipulation, but collecting high-quality contact-critical demonstrations remains difficult with low-cost teleoperation hardware. We present ViHaTeleop, a lightweight (0.7 kg), low-cost (\$550) visual-haptic teleoperation system with SLAM-based wrist tracking, camera-based hand tracking, and finger-wise vibrotactile feedback through Linear Resonant Actuators (LRA). The system includes several design choices (LED illumination, fisheye hand camera, and tactile-aware retargeting constraints) and is deployed on Franka + LEAP Hand + 9DTact in both real and simulated environments. Under matched with/without-haptic conditions with nine participants across six contact-critical tasks, haptics improved success rates across all tasks (+2.2 to +15.6 percentage points), while completion-time effects were task-dependent. Subjective ratings showed significant gains in contact clarity and grasp confidence in both simulation and real-world settings (Wilcoxon signed-rank, $p<0.05$). We also integrate a lightweight depth-camera-based tactile proxy in Isaac Sim, enabling a full pipeline from multi-modal demonstration collection to visual-tactile policy training. Preliminary downstream validation by training visual-tactile policies from collected demonstrations shows tactile cues benefit contact-critical subtasks (peg-in-hole: +17 percentage points over vision-only).

0.63

Towards Physical Underwater Robotic Assistance for Scuba Diver Movement in Confined Spaces

cs.RO ·2026-10-01 ·Demetrious T. Kutzke, Junaed Sattar
similarity 0.59
Abstract

Scuba divers are taught to control their depth to avoid rapid ascents and descents, which could result in serious injuries such as gas embolisms and barotrauma. However, many underwater tasks necessitate lateral control, maintaining distance between subsea structures such as coral reefs, submerged drilling instrumentation, or unexploded ordnance. In this work, we discuss a first-of-its-kind wearable robotic solution providing thruster-actuated directional guidance to a diver, as distinct from prior propulsive-assistance exoskeletons. We introduce ``Robotic Assisted Diver Movement in Confined Spaces'' (RADMCS), a wearable robot that assists divers in maintaining a fixed distance from subsea structures by leveraging perception techniques in monocular depth estimation and force-feedback from submersible thrusters to provide haptic feedback. Its small and compact form factor creates a foundational platform that could be expanded to include more sophisticated control and navigation behaviors. We present results from Institutional Review Board (IRB) in-water studies with eight human scuba diver participants on threshold sensitivity tests in both a closed-water swimming facility and ocean environments; distance-maintaining experiments in a closed-water facility; and form, fit, and function testing in the ocean. We demonstrate that relatively low thrust values (10 percent of maximum) allow robotic direction of a human's movement using the physical sensation of the robot's guidance.

0.19