CS Paper Radar

Passthrough Rigidity: The Behavioral and Visuomotor Costs of Mediated Perception

Markus D. Solbach, Mohit Goyal, Sakar Khattar, Jayant Varma, Bjorn Vlaskamp, John K. Tsotsos, Konstantine Tsotsos

cs.HC ·submitted 2026-09-21 17:59 UTC · v1
publishedcode/data

Abstract

Broad public adoption of head-mounted displays using video passthrough remains elusive despite significant market investment. A precise understanding of why users experience persistent discomfort even as hardware factors such as resolution and latency have dramatically improved remains an open issue. This paper investigates the impact of viewing the world through video passthrough systems on human behavioral and physiological patterns through a large-scale multimodal study. We developed a novel protocol to capture synchronized oculomotor, kinematic, and physiological data during a block assembly task requiring complex hand-eye coordination. Using a within-subject design (N=110), we evaluated both natural and passthrough viewing conditions. Our results reveal a four-fold suppression of rotational head velocity and a pronounced decoupling of head-gaze coordination. This suggests motor caution being employed as an adaptive strategy - which we term "Passthrough Rigidity". This phenomenon appears to shift the information-gathering burden to the oculomotor system, resulting in significantly longer fixation durations and restricted visual search patterns. These kinematic shifts directly correlate with poorer task performance and measurable physiological cost, evidenced by a significant reduction in blink duration and increased reports of ocular strain and cognitive load. We conclude that current passthrough implementations induce a measurable shift from flexible exploration to motor caution, where task performance is preserved at the cost of user comfort and biomechanical efficiency. These findings provide a novel quantitative framework for evaluating and improving future XR devices, establishing that resolving "comfort" for passthrough requires addressing the deep-seated biomechanical compensations caused by mediated perception. Data available at https://data.nvision.eecs.yorku.ca/Passthrough

Comment
10 pages, 16 Figures, ISMAR

Closest papers

Nearest neighbours by meaning across the whole index.

Gaze Under Strain: How Oculomotor Behavior Tracks Cybersickness Progression in VR

cs.HC ·2026-10-23 ·Nevzat Umut Demirseren, Mehedi Hasan Raju, Oleg Komogortsev, Isayas Berhe Adhanom
published similarity 0.72
Abstract · Accepted for publication at ACM VRST 2026. 10 pages, 4 figures

Cybersickness remains a major barrier to comfortable, accessible virtual reality (VR). Arising from mismatches between visual and vestibular signals, it places sustained demands on gaze stabilization and visual processing. Because gaze is the mechanism through which users actively sample the virtual environment, cybersickness-induced disruptions may alter both what users perceive and how discomfort itself emerges. Yet, how eye-movement behavior evolves alongside discomfort during prolonged VR exposure remains poorly understood. We present a behavioral analysis of eye-tracking data from 19 participants navigating a realistic virtual environment over a 20-minute session, with discomfort rated repeatedly throughout the experiment. Linear mixed-effects models examined gaze constructs spanning fixation stability, visual exploration, and gaze allocation as functions of discomfort, exposure duration, and their interaction. Our results show that fixation stability degraded progressively with increasing discomfort, with the effect strengthening over time. Visual exploration showed no marginal discomfort effect but exhibited significant discomfort-by-time interactions, indicating that saccadic behavior adapts dynamically rather than responding uniformly to discomfort. Gaze allocation was not associated with discomfort but shifted systematically with time. These findings show that cybersickness does not uniformly disrupt gaze. Instead, different aspects of eye-movement behavior respond differently and change in ways that only become visible when discomfort is tracked repeatedly over time. Understanding how gaze evolves during cybersickness progression provides the behavioral grounding needed to better interpret how discomfort shapes the user's visual experience of immersive environments and to design more sensitive detection approaches.

0.32

Looking Around by Looking Around: Omnidirectional Gaze-based VR Viewport Control

cs.HC ·2026-08-30 ·Hock Siang Lee, Jinghui Hu, Florian Weidner, Haopeng Wang, Hans Gellersen
published similarity 0.70
Abstract

Traditional VR viewport control primarily relies on head and torso movement, which can be effortful and limiting in both constrained and extended-use settings. We introduce Looking Around by Looking Around (LALA), a gaze-based VR pitch-and-yaw viewport control technique designed for natural and effortless omnidirectional exploration via eye movements, without requiring or obstructing movement of the head, hand, or body, offering a low-effort and highly accessible interaction method. Because gaze is primarily used for perception and exhibits oculomotor and perceptual asymmetries, using it directly for control is difficult. To address this, we designed an asymmetric omnidirectional control profile for the eye, then built on it to exploit tendencies for eyes to stay within comfortable regions for viewport control. We evaluated LALA in a user study (N=18) featuring two contrasting tasks: alignment towards known directions and open-ended visual search towards unknown directions. LALA was strongly preferred over the traditional baseline, achieving competitive performance while enabling fully hands-free interaction with minimal physical movement.

0.36

Unpacking the process of watching a 360-degree video: How physiological responses and head rotation relate to affective and cognitive outcomes

cs.HC ·2026-10-30 ·Portia Wang, Yujie Tao, Monique Santoso, Jeremy Bailenson
similarity 0.64
Abstract

A defining feature of 360-degree video is that the perceptual experience is actively constructed. As viewers turn their heads, shift their gaze, and encounter changing visuals, their perceptual experience is not predetermined, but rather emerges through ongoing exploration. Yet, it is unclear how the process of watching immersive videos unfolds, and how it is related to psychological and physiological outcomes. We present a large-scale study (N=200) in which participants experienced one of 10 pieces of 360-degree videos in virtual reality. Results showed that greater head rotation was associated with greater valence, higher attention focus, longer length of subjective review, and lower simulator sickness; greater pupil dilation was associated with lower valence, lower review ratings, and greater use of negative tone in written reviews. A mediation analysis revealed that participants who rotated their heads more experienced greater frame-to-frame pixel-level perceptual changes in their field of view, which in turn resulted in greater valence and lower simulator sickness. Moreover, participants tended to rotate their heads toward field of view regions with greater perceptual movement, while directing their eye gaze toward less perceptually dynamic areas within that view. We outline theoretical and practical contributions for designing 360-degree content and predicting experiential outcomes.

0.14

Effects of Locomotion and Viewpoint Control on Immersive Robot Telepresence for Navigation and Inspection

cs.RO cs.HC ·2026-11-18 ·Ammaar Zaman, Neven ElSayed, Oliver Prentner, Mohamed Ellethy, Mostafa M. Lotfy +2 more
similarity 0.63
Abstract

In many robot telepresence interfaces, movement and viewing are managed through handheld inputs. This can divide attention between steering, looking, and inspection. We compared joystick versus omnidirectional treadmill locomotion and joystick-based versus head-guided camera control in a within-subject 2x2 study in which 17 participants used a virtual reality headset to guide a quadruped robot through a physical maze while locating marked wall symbols and capturing them as visual evidence. Mixed-effects analyses showed that head-guided camera control was associated with faster completion, lower wall-contact cost, lower workload, and more favorable presence, usability, technology acceptance, and flow than joystick-based camera control. The strongest overall objective performance was observed in the joystick locomotion plus head-guided camera condition, whereas the omnidirectional treadmill plus head-guided camera condition emerged as the strongest experiential compromise. Exploratory camera analyses showed that head-guided camera control reduced switching, increased reliance on the pan-tilt view, and supported a more stable viewing strategy across the maze. These strategy differences were consistent with the performance advantage, especially for wall-contact duration. Targets captured remained near ceiling and simulator sickness remained low across all conditions. Overall, the results indicate that viewpoint control was the stronger differentiator in this combined navigation and inspection task, while locomotion still shaped the objective and experiential tradeoff.

0.29

Beyond the Mirror: Balancing Interaction Modality and Avatar Fidelity in Public 3D Virtual Try-On Systems

cs.HC ·2026-08-24 ·Yueqian Guo, Tianzhao Li, Xin Lv
similarity 0.62
Abstract

Virtual Try-On (VTON) systems deployed on large public displays face a dual barrier: the physical strain of mid-air interaction and the social inhibition caused by public self-consciousness. This paper presents a real-time 3D avatar system integrating markerless motion capture with dynamic visual fidelity control to investigate and mitigate both barriers. Through a dual-study empirical evaluation, we first decoupled physical fatigue from gesture interaction ($N=20$), demonstrating that interaction fatigue is primarily driven by visuomotor latency rather than the physical act of gesturing; our optimized low-latency gesture pipeline achieved usability comparable to touchscreens while delivering superior immersion and hygiene. Building on these insights, our second study ($N=25$) investigated the "avatar fidelity paradox" via a $2 \times 2$ factorial design manipulating interaction modality (gestures vs. touch) and visual fidelity (photorealistic MetaHuman vs. stylized mannequin). Results reveal that while high fidelity and mid-air gestures independently maximize virtual embodiment ($p < .05$), their combination elicits the highest social awkwardness. Crucially, low-fidelity avatars serve as a "psychological mask" that alleviates public embarrassment during expressive gestures, while mid-air gestures simultaneously act as a compensatory mechanism to preserve perceived try-on trust despite reduced visual realism. Finally, we propose a context-aware fidelity framework to balance privacy, immersion, and commercial trust in public spatial interactions.

0.08

GAZEleak: Passcode Inference Against Eye-tracking XR Devices Through External Observation

cs.CR ·2026-09-28 ·Hwanjo Heo, Junhee Lee, Jinwoo Kim
similarity 0.61
Abstract · The vendor requests more time to review our responsible disclosure

Mixed-reality headsets such as Apple Vision Pro replace the touch screen with gaze-as-pointer interaction: the wearer looks at a target and confirms with an air pinch. Because the display is inside the headset and the eye tracker is walled off from third-party software, such input is widely assumed to be unobservable to bystanders---a built-in defense against the shoulder-surfing that plagues phones and laptops. We present GAZEleak, a side-channel attack that recovers gaze-driven input from external video of head motion alone, under a strictly local, physical-observer threat model: the adversary only films the wearer from across the room and installs no software on the device. The attack exploits the centrally coupled eye-head motor program: gaze shifts recruit small, target-dependent head reorientations that project into sub-degree pose changes recoverable from commodity video. GAZEleak implements a measurement-based, sparse-optical-flow inference pipeline for users' 6-digit device passcodes. On a preliminary front-view dataset from three author-subjects who were aware of the attack hypothesis, GAZEleak places the true code within the top ten guesses for 56% (10 of 18) of test codes under a cross-person protocol with no labeled victim data, and for every code of the most exposed subject. The performance is subject-dependent: no passcode from the least exposed subject reaches the top ten, although its median guessed passcode rank is 12,786 rather than 500,000 expected from an uninformative ordering. These results provide preliminary evidence that gaze-coupled head motion can expose passcode information under controlled conditions, while motivating broader evaluation across users, behaviors, and capture settings.

0.26

EgoWholeView: Camera-Only Head-Mounted Capture of Body and Bimanual Hand Motion for Embodied Demonstrations

cs.CV ·2026-10-12 ·Yaoqing Hu, Xuguo He, Daohan Cui, Yukun Shi, Kai Chen
benchmark similarity 0.60
Abstract · 17 pages, 9 figures

Head-mounted body and bimanual motion capture supports self-avatars, teleoperation, and embodied demonstrations. However, a single egocentric camera is vulnerable to a limited field of view, scale ambiguity, and hand-object occlusion. We present EgoWholeView, a wearable capture system of six synchronized wide-angle RGB cameras with a role-specialized pipeline. The outward views localize the rig in a world frame, while an overlapping downward pair independently predicts SMPL-X body and MANO hand parameters. Both predictors are adapted to egocentric fisheye data. To combine the two, reliability cues measured from the predictions themselves weight the fusion of body and hand joint rotations, while shape is aggregated across views and frames. The pair's baseline fixes metric scale, and triangulation recovers root translation. Relative to monocular prediction, fusion reduces PA-MPJPE by 3.7% for the body and 11.2% for the hands, and temporal refinement lowers angular-acceleration magnitude by 32.2% and 31.0%, respectively. On our six-camera recording, the pipeline produces body and hand states in a shared world frame using benchmark-fitted gates without retuning, and temporal refinement lowers hand acceleration magnitude by 41.08%. Using only RGB cameras and no third-person view, EgoWholeView offers a low-cost, portable route to egocentric interaction data.

0.35

Feel My Focus: Gaze-Triggered Digital Twin Creation for Remote Collaboration

cs.HC cs.CVcs.GR ·2026-11-08 ·Shih-Yu Ma, Suibi Che-Chuan Weng, Ming-Yuan Pai, Tom Yeh, Ellen Yi-Luen Do
top venue similarity 0.60
Abstract · 5 pages, 3 figures

We present Feel My Focus, a gaze-triggered mixed reality (MR) demonstration that enables rapid creation and sharing of lightweight 3D digital twins during remote collaboration. Unlike systems that use gaze primarily as a referential pointer, Feel My Focus treats sustained visual attention as a generative action: a fixation on a physical object initiates single-image capture, attention-guided segmentation, and one-shot 3D reconstruction, allowing a remote partner to inspect and discuss the object in situ. The system integrates gaze-tracking glasses, modular segmentation and reconstruction services, and an MR client for runtime streaming and interaction. This design prioritizes responsiveness and experiential continuity over photometric fidelity. We report observations from a public demonstration, highlighting design trade-offs, failure modes, and user control considerations for deploying gaze-triggered generative systems in real-world collaborative settings.

0.32