CS Paper Radar

Online Multimodal Workload Assessment in Contact-Rich Physical Human-Robot Interaction

Yanyi Chen, Fan Yang, Min Deng

cs.RO ·submitted 2026-09-16 02:30 UTC · v1

Abstract

Contact-rich physical human--robot interaction (pHRI) imposes time-varying demands associated with physical interaction, motor regulation, and physiological response, motivating continuous assessment of interaction workload. This paper presents an online multimodal assessment framework that integrates interaction wrench, planar tool-center-point (TCP) kinematics, and skin conductance level (SCL) into four interpretable workload-related factors. Their relative contributions are adjusted using path curvature to reflect changes in motion demand and task progression to account for gradual physiological variation over time. The framework was evaluated with 24 participants across 18 controlled combinations of temperature, acoustic noise, and illuminance under two admittance-control modes. Strict leave-one-subject-out (LOSO) evaluation used standardized pupil diameter ($\mathrm{PD}_z$) as an independent physiological reference and included comparisons with static variants and representative state-of-the-art learning-based baselines. The proposed framework achieves a cohort-mean $30\,\mathrm{s}$ block-wise Spearman correlation of $ρ_{30}=0.308$ with the physiological reference, with positive subject-level correspondence in 23 of 24 participants. Its overall performance is comparable to the state-of-the-art learning-based baseline. At the same time, our framework keeps the assessment process transparent through explicit workload-related factors and defined weighting rules, while outperforming the corresponding fixed-weight formulation. The framework also maintains consistent performance across the two tested admittance-control modes. These results support a transparent and interpretable approach to continuous interaction workload assessment in contact-rich pHRI.

Closest papers

Nearest neighbours by meaning across the whole index.

Establishing a Dynamic Multimodal HRI Dataset for Engagement Analysis with a Humanoid Robot

cs.RO ·2026-09-03 ·Buwan Kim, Wonse Jo
similarity 0.63
Abstract

This paper presents an experimental design for constructing a multimodal dataset to analyze user engagement in human-robot interaction (HRI). Prior studies have mainly relied on observable behavioral cues, with limited frameworks integrating physiological signals. We therefore propose a structured data-collection protocol to build a multimodal dataset that includes wearable physiological signals, behavioral data, and self-report measures under different levels of task complexity defined in this experiment.

0.09

A Vision Based Framework Integrating Attention and Action Cues for Interpretable Cognitive Workload Assessment in Human-Robot Collaborative Assembly

cs.RO ·2026-09-14 ·Junyan Xiong, Naiyi Feng, Xingke Xia, Qihang Fan, Suchang Chen +1 more
similarity 0.61
Abstract · 42 pages, 11 figures

The introduction of human-robot collaboration (HRC) in industrial assembly operations is revolutionizing the manufacturing landscape. In this evolving environment, operators are required to seamlessly coordinate their manual tasks with real-time task information and robotic behaviors. These demands fluctuate during operation, yet conventional workload assessments depend on body-worn physiological sensors that complicate practical deployment. Here, we present a vision-based attention--action framework for continuous and interpretable workload-related assessment in HRC assembly. The framework combines RGB-D observations with robot states and calibrated task-related areas to construct a temporally confirmed representation of operator behavior. This representation identifies where task demand is concentrated and explains how it develops when attention and action diverge, the task context changes, or the operator hesitates. We evaluated the framework in a three-level collaborative gearbox assembly experiment with ten participants, using subjective ratings and synchronized physiological signals as independent references. Raw NASA-TLX ratings confirmed increasing perceived workload across conditions, with significant effects on overall workload and its mental and temporal dimensions. The vision-derived HRC-CWL output was significantly associated with ECG-derived features in seven of nine participants with complete correlation data. Synchronized interaction episodes further showed temporal correspondence between detected hesitation and physiological activity. Real-time deployment demonstrated that the framework can operate without requiring operators to wear additional sensors. These findings support HRC-CWL as an interpretable behavioral proxy for cognitive ergonomics analysis and adaptive robot assistance, rather than a direct psychophysiological measure of workload.

0.29

OpenZilo: An Open Smart-Ring Platform for Human-Centric Sensing and Interaction with Robots

cs.RO cs.HC ·2026-11-02 ·Qianggang Ding, Xingyao Wang, Rui Feng, Chi Qin, Weijian Lin +5 more
similarity 0.59
Abstract · 17 pages, 5 figures, 8 tables

Wearable robot interfaces must connect user input to timely physical action. We present the proposed platform, an open smart-ring platform for studying this connection across task stages. The ring integrates a microphone, an inertial measurement unit, photoplethysmography, and skin-temperature sensing. Its software architecture separates acquisition, event interpretation, interaction policy, and robot execution. We characterize seven operating modes and evaluate speech and gesture control with 10 participants. Across five commands and 250 trials per modality, speech and gestures achieved task success rates of 90.4% and 88.8%, respectively. Recognition accuracy exceeded task success for both modalities. Gaussian hidden Markov models achieved higher accuracy than a support vector machine across three vocabulary sizes, with longer scoring times. Following a shared spoken task start, gesture-based adjustments had shorter median response times than speech. Whole-task success was 70.0% with gestures and 64.0% with speech. We further examine physiological-event-triggered robot interaction using simulated event notifications. Mean notification-receipt-to-arrival times were 11.71 s for the robot and 9.20 s for an on-site person. These descriptive findings support evaluating wearable robot interfaces through both recognition and physical outcomes, with timing measures appropriate to each task stage.

0.32

A Physics-Consistent Benchmark for Contact-Rich Human-Robot Interaction in Assistive Care

cs.RO ·2026-09-02 ·Chengxiao He, Shanghai Yuan, Liuqun Fan, Shenzhen Zhu
benchmark similarity 0.59
Abstract · 8 pages, 4 figures. Submitted to the 2026 IEEE International Conference on Robotics and Biomimetics (ROBIO)

Conventional task-level evaluation asks whether a robot policy completes a specified action, but can miss failures that emerge only during physical human contact. This limitation is critical in contact-rich assistive tasks, where meaningful evaluation requires a physically responsive human, interaction-quality assessment beyond task success, and a leak-free observer-scorer protocol. We introduce a physics-consistent benchmark for contact-rich human-robot interaction, instantiated in robot-assisted bathing. The benchmark combines a deformable, passively responding human, physics-aware scores alongside task-level success, and a frozen vision-only / scorer-only evaluation protocol. To establish physical validity, region-wise simulated responses are calibrated against force-indentation measurements from Franka impedance pushes on a medical-care manikin. Under a frozen T1-T7 protocol with 140 runs per method, an LLM-augmented state machine (State Machine) achieves 72.9% task success but drops to 56.4% after correct-region and force-safety screening; VoxPoser produces lighter and more stable contact but completes only 27.9% of trials; and zero-shot pi0.5 achieves 0.7% task success with no correct-region or safety-gated successes. These results show that task completion alone does not imply physically valid contact and motivate physics-aware screening before deployment of contact-rich assistive robot policies.

0.14

Exploring Mixed Reality for Engagement in Robotics: A Comparison of Robot Control Methods

cs.HC ·2026-10-12 ·Samantha Edvell, Dinh Tung Le, Gavin Paul
similarity 0.59
Abstract · 10 pages. Submitted to ACRA 2026

This paper presents a Mixed Reality (MR) robotics experience that provides an accessible introduction to interacting with a physical robot. The application combines a TurtleBot with a virtual animal-tending activity and three control methods: Joystick, Waypoint, and Path-Drawing. A mixed-methods user study with 10 participants investigated changes in robotics interest and confidence following the experience and how the control methods influenced engagement, workload, and preference. Confidence increased following the experience, with the exploratory analysis indicating a significant pre--post difference, while the increase in interest was not statistically significant. The control methods produced clear differences in perceived workload and ease of use, with Path-Drawing generally more demanding and difficult to use. Joystick was preferred by eight of ten participants and was associated with better perceived performance and lower frustration. However, no statistically significant differences were detected in the other engagement dimensions, suggesting that the most preferred method was not necessarily more engaging across all aspects of the experience. These findings highlight the importance of considering both usability and engagement when designing how users interact with robots in introductory MR experiences.

0.23

When Should Robots Intervene? Balancing Engagement and Intrusiveness in Human-Robot Interaction

cs.RO ·2026-09-18 ·Lavinia Hriscu, Valerio Bo, Alberto Sanfeliu, Anaís Garrell
published similarity 0.58
Abstract · Accepted for publication at the 35th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2026)

Designing effective Human-Robot Interaction in task-oriented settings requires carefully balancing user engagement with socially acceptable levels of robot intrusiveness. In this paper, we examine how different robot intervention strategies shape user experience, interaction dynamics, perceived intrusiveness, and sense of support. We compare two approaches: a continuous engagement-seeking robot strategy, and a context-aware strategy that selectively intervenes based on the user's state and task context. Both approaches rely on multimodal behavioral cues, including body orientation and attentional signals, to guide robot actions. We evaluate these strategies in a user study with 32 participants performing a task in a simulated hospital environment. Our findings show that higher interaction frequency does not necessarily lead to better engagement. Instead, we observe a systematic trade-off between perceived support and intrusiveness, influenced by factors such as physical proximity and user effort. These results provide empirical evidence that effective engagement in HRI depends on adaptive, context-sensitive intervention policies.

0.34

The Role of Variability in Human-Machine Interaction Experience

cs.HC eess.SY ·2026-08-11 ·Sean Kille, Jan Lars Hagemann, Anne Voormann, Balint Varga, Andrea Kiesel +1 more
similarity 0.58
Abstract · 19 pages, 7 figures

Human-machine interaction (HMI) requires control strategies that account for the nature of human motor behavior. Conventional shared-control and haptic-assistance methods typically ignore the stochastic nature of human behavior, potentially limiting both performance and human interaction experience. In this study, we designed an experimental setting and evaluated a novel human-variability-aware optimal controller. Participants performed a physically coupled haptic interaction task in three conditions: a controller mode that aims at conventionally reducing overall variability, a variability-aware controller mode designed to maintain human natural variability patterns, and a human-only control condition serving as a baseline. We analyzed behavioral variability, task performance, and human interaction experience. The results show that considering natural movement variability significantly increased perceived interaction quality in terms of usability while maintaining task performance. These findings highlight the importance of incorporating stochastic human movement characteristics into shared-control designs and demonstrate the feasibility and benefits of the proposed control strategy for human-centered control design of HMI.

0.10

M3-Tele: A Unified Multimodal Teleoperational Framework for Compliant Whole-Body Mobile Manipulation

cs.RO ·2026-09-07 ·Hengxiang Chen, Shenwen Deng, Yujian Ma, Gan Ma, Qiang Li +1 more
similarity 0.58
Abstract · 15 pages,12 figures,a under-review journal

Executing contact-rich tasks efficiently requires the seamless integration of whole-body coordination and physical compliance regulation. However, existing teleoperation and data-collection frameworks often overlook the joint consideration of multimodal perception and coordinated whole-body operation. This limitation can reduce the efficiency and quality of demonstration collection, thereby affecting the effectiveness of downstream policy learning. In this work, we present \textbf{M3-Tele}: A Unified \underline{M}ultimodal \underline{Tele}operational Framework for Compliant Whole-Body \underline{M}obile \underline{M}anipulation, enabling stable physical interaction and capturing aligned visual, tactile, force, and proprioceptive observations during task execution. Extensive experiments demonstrate that the proposed framework significantly improves contact-rich teleoperation performance. The proposed controller reduces the force tracking error from 4.132~N to 0.346~N, the contact loss from 2.46 to 0.02 events per trial and the tactile deformation error by 65\%. User studies across four mobile manipulation tasks also verify the reliability and usability of the proposed system. Furthermore, Diffusion Policy experiments highlight the value of joint tactile and force sensing.

0.22