CS Paper Radar

A Biomimetic Gaze Control to Restore Head-Neck Movements

Colin Rubow, Haohan Zhang

cs.RO ·submitted 2026-10-02 20:28 UTC · v1
top venue

Abstract

Powered neck exoskeletons are developed to restore head-neck motions for those with neck muscle weakness. Current methods using hand-held input devices to control these robotic systems are unintuitive or non-viable for these populations. Alternatively, the eyes coordinate with the head to accomplish many daily tasks. Therefore, movements of the eyes can be a useful signal to control the neck exoskeletons without needing to use the hands. However, previous control models lack biobehavioral basis as well as means to personalize the control behavior, causing unnatural head-neck movements. This paper presents a novel eye-tracking controller design leveraging the vestibulo-ocular reflex to emulate natural eyehead coordination. Personalization tools were also designed, consisting of sliders to allow users to adjust the parameters of this controller. Here, we present the technical design and validation of this system in human users. Compared to the previous eye-tracking model, the new controller provides natural head motions personalized to the user through more natural velocity profiles and a 24 percent average increase of accurate motion intent detection. This system provides a powerful platform for studying eye-tracking controllers in the context of personalization and preference, and ultimately restoring the lost head-neck mobility in patient populations.

Closest papers

Nearest neighbours by meaning across the whole index.

Looking Around by Looking Around: Omnidirectional Gaze-based VR Viewport Control

cs.HC ·2026-08-30 ·Hock Siang Lee, Jinghui Hu, Florian Weidner, Haopeng Wang, Hans Gellersen
published similarity 0.62
Abstract

Traditional VR viewport control primarily relies on head and torso movement, which can be effortful and limiting in both constrained and extended-use settings. We introduce Looking Around by Looking Around (LALA), a gaze-based VR pitch-and-yaw viewport control technique designed for natural and effortless omnidirectional exploration via eye movements, without requiring or obstructing movement of the head, hand, or body, offering a low-effort and highly accessible interaction method. Because gaze is primarily used for perception and exhibits oculomotor and perceptual asymmetries, using it directly for control is difficult. To address this, we designed an asymmetric omnidirectional control profile for the eye, then built on it to exploit tendencies for eyes to stay within comfortable regions for viewport control. We evaluated LALA in a user study (N=18) featuring two contrasting tasks: alignment towards known directions and open-ended visual search towards unknown directions. LALA was strongly preferred over the traditional baseline, achieving competitive performance while enabling fully hands-free interaction with minimal physical movement.

0.36

Determination of the Human-Robot Interaction Workspace for Robot Assisted Head Stabilization Control

cs.RO physics.bio-phphysics.med-ph ·2026-10-19 ·Xinmin Liu, Alexander L. Van Slyke, Destinee Holloman, Ahmad Sakaamini, Rodney D. Wiersma
similarity 0.61
Abstract

Accurate head immobilization is critical for medical procedures such as neuroimaging and intracranial radiation therapy, where even minor motion can compromise precision. Traditional methods, such as rigid stereotactic frames and thermoplastic face masks, are invasive, uncomfortable for patients, and require a time-consuming setup. To overcome these limitations, a novel approach employs a 6-degree-of-freedom (6DoF) robotic head holder that is integrated with real-time optical motion tracking, enabling dynamic compensation for head movements without physical restraint. However, implementing this system requires a detailed biomechanical understanding of skull-cervical spine interactions to define safe and effective motion limits. In this study, we conduct a human volunteer investigation using a 6DoF Stewart platform robot to characterize biokinematic workspace of the head-neck system and establish foundational data for human-robot interaction in precision head stabilization.

0.21

GazeAgent: VLM-Guided Implicit Gaze Calibration from Egocentric Multimodal Behavior for AI Glasses

cs.HC cs.CV ·2026-10-17 ·Yuhong Zhang, Khanh An Nguyen, Xinyu Tian, KungYi Chang, Abraham Padua +4 more
similarity 0.59
Abstract · 27 pages, 14 figures, 3 tables

AI glasses typically require explicit, user-specific eye-tracker calibration, and accuracy can drift when the device shifts during use. We present GazeAgent, a VLM-guided framework that continuously and adaptively improves gaze-calibration accuracy from naturally occurring behavior without asking users to interrupt their activities. GazeAgent identifies precise visual attention (PVA) events from multimodal streams of inertial measurement unit (IMU), gaze, and wearer-hand signals, validates their task relevance, and localizes functional targets as pseudo-calibration targets. In a 10-participant scripted study, the system detected all 820 predefined PVA opportunities and, under device disturbance, reduced median gaze error by 44.1% compared with explicit calibration. In a five-participant case study, autonomous calibration produced the lowest workload and interruption ratings, and all participants preferred it for routine use. By deriving task-critical targets from natural multimodal behavior, GazeAgent demonstrates the potential for continuous, adaptive, and low-burden gaze interaction on egocentric AI glasses.

0.08

Passthrough Rigidity: The Behavioral and Visuomotor Costs of Mediated Perception

cs.HC ·2026-09-21 ·Markus D. Solbach, Mohit Goyal, Sakar Khattar, Jayant Varma, Bjorn Vlaskamp +2 more
publishedcode/data similarity 0.59
Abstract · 10 pages, 16 Figures, ISMAR

Broad public adoption of head-mounted displays using video passthrough remains elusive despite significant market investment. A precise understanding of why users experience persistent discomfort even as hardware factors such as resolution and latency have dramatically improved remains an open issue. This paper investigates the impact of viewing the world through video passthrough systems on human behavioral and physiological patterns through a large-scale multimodal study. We developed a novel protocol to capture synchronized oculomotor, kinematic, and physiological data during a block assembly task requiring complex hand-eye coordination. Using a within-subject design (N=110), we evaluated both natural and passthrough viewing conditions. Our results reveal a four-fold suppression of rotational head velocity and a pronounced decoupling of head-gaze coordination. This suggests motor caution being employed as an adaptive strategy - which we term "Passthrough Rigidity". This phenomenon appears to shift the information-gathering burden to the oculomotor system, resulting in significantly longer fixation durations and restricted visual search patterns. These kinematic shifts directly correlate with poorer task performance and measurable physiological cost, evidenced by a significant reduction in blink duration and increased reports of ocular strain and cognitive load. We conclude that current passthrough implementations induce a measurable shift from flexible exploration to motor caution, where task performance is preserved at the cost of user comfort and biomechanical efficiency. These findings provide a novel quantitative framework for evaluating and improving future XR devices, establishing that resolving "comfort" for passthrough requires addressing the deep-seated biomechanical compensations caused by mediated perception. Data available at https://data.nvision.eecs.yorku.ca/Passthrough

0.29

An Eye-Tracking Dataset for Viewing Distance Categories in Real-World Scenarios

cs.HC ·2026-08-29 ·Dohwa Kim, Yejin Choi, Seungbok Lee, Chi Yoon Jeong, Eunji Park
dataset similarity 0.59
Abstract · 22 pages, 6 figures, Manuscript submitted to Scientific Data

Estimating viewing distance from gaze behavior is essential for understanding user intent and enabling distance-aware interactive systems. However, most existing eye-tracking datasets have been collected in constrained settings, such as laboratory environments or static tasks. Consequently, they only partially capture viewing behaviors in real-world situations where viewing distance changes with natural head and body movements. We introduce GazeDepth, an eye-tracking dataset collected from 19 participants using a wearable tracker during tasks reflecting real-world scenarios. GazeDepth includes fixed-distance viewing scenarios with constant observer-target distances at near (33 cm), middle (50 cm), and far (300 cm), as well as variable-distance viewing scenarios in which participants shift gaze among targets at different depths in indoor and outdoor environments. The dataset provides synchronized gaze data, pupil size, 3D eye-vectors, and head-motion signals, along with distance labels. Statistical analyses showed that distance-related gaze features, such as vergence angle and estimated viewing distance, differed consistently across viewing-distance categories. In addition, classification models trained on GazeDepth further demonstrated that the dataset captures gaze characteristics that distinguish the three viewing-distance categories, supporting gaze-based distance inference and distance-aware interaction in realistic scenarios.

0.26

Adapting Appearance-Based Gaze Estimation to Narrow-Range, Long-Duration Screen Viewing

eess.IV ·2026-10-08 ·Jordan Prescott, Kleanthis Avramidis, Shrikanth Narayanan
benchmark similarity 0.58
Abstract

Appearance-based gaze estimation offers a low-cost alternative to infrared eye tracking for screen-based behavioral and clinical applications, however existing models are typically developed for wide ranges of gaze angle and head pose. Prolonged screen viewing presents a distinct regime in which gaze remains near the screen center, head motion is limited, and calibration drift accumulates over time. In this work, we benchmark six published estimators along with a proposed differential-gaze model on 140 long-duration facial video recordings with synchronized eye tracking under subject-disjoint evaluation and a common budget of calibration frames. Differential estimation was the only static method that significantly improved upon a baseline predictor of each recording's mean gaze, and was further boosted by addition of temporal context. It also yielded the strongest agreement with reference fixations and saccades, demonstrating that low angular error alone could not validate eye movement reconstruction. These findings establish differential estimation as a promising foundation for reliable gaze tracking in long-duration, narrow-range settings.

0.07

GAZEleak: Passcode Inference Against Eye-tracking XR Devices Through External Observation

cs.CR ·2026-09-28 ·Hwanjo Heo, Junhee Lee, Jinwoo Kim
similarity 0.58
Abstract · The vendor requests more time to review our responsible disclosure

Mixed-reality headsets such as Apple Vision Pro replace the touch screen with gaze-as-pointer interaction: the wearer looks at a target and confirms with an air pinch. Because the display is inside the headset and the eye tracker is walled off from third-party software, such input is widely assumed to be unobservable to bystanders---a built-in defense against the shoulder-surfing that plagues phones and laptops. We present GAZEleak, a side-channel attack that recovers gaze-driven input from external video of head motion alone, under a strictly local, physical-observer threat model: the adversary only films the wearer from across the room and installs no software on the device. The attack exploits the centrally coupled eye-head motor program: gaze shifts recruit small, target-dependent head reorientations that project into sub-degree pose changes recoverable from commodity video. GAZEleak implements a measurement-based, sparse-optical-flow inference pipeline for users' 6-digit device passcodes. On a preliminary front-view dataset from three author-subjects who were aware of the attack hypothesis, GAZEleak places the true code within the top ten guesses for 56% (10 of 18) of test codes under a cross-person protocol with no labeled victim data, and for every code of the most exposed subject. The performance is subject-dependent: no passcode from the least exposed subject reaches the top ten, although its median guessed passcode rank is 12,786 rather than 500,000 expected from an uninformative ordering. These results provide preliminary evidence that gaze-coupled head motion can expose passcode information under controlled conditions, while motivating broader evaluation across users, behaviors, and capture settings.

0.26

Eye, Robot: Learning Expressive Gaze Dynamics via a VLM-Guided Transformer-VAE

cs.RO cs.AI ·2026-10-14 ·Yuhang Hu, Junlin Lyu, Yuan Fang, Xiaoying Shen, Lechen Zhang +4 more
benchmark similarity 0.58
Abstract

Gaze coordinates attention in human interaction through coupled eye, head, and eyelid motion. A humanoid robot must choose where to look while generating a suitable movement toward that target; fixed tracking rules alone do not model the temporal structure of human gaze. We separate these tasks with an asynchronous vision-language model that selects a scene referent, a reactive perception module that maintains fixation, and a Realistic Gaze Transformer (RGT) that generates target-conditioned motion. RGT is a Transformer-VAE trained on markerless human recordings retargeted to the robot's 11-DoF joint space: self-supervised pretraining encodes recent full-head history, and autoregressive fine-tuning exposes the model to its own predictions during rollout. At inference, a four-dimensional binocular-eye trajectory conditions coordinated neck and eye motion; an auxiliary blink predictor triggers eyelid templates during playback. On a two-second benchmark in which all controllers receive the same ground-truth target, RGT-Base obtains the lowest reported overall seven-joint MSE, and physical demonstrations show the integrated targeting and control pipeline. The evidence establishes robot-space kinematic fidelity and real-time RGT inference; perceived naturalness and open-vocabulary target-selection accuracy remain questions for controlled evaluation.

0.14