CS Paper Radar

PlanePivoting: Exploration and Optimization of Gaze-Mouse Cursor Alignment for Spatial Object Translation

Jinwook Kim, Sangmin Park, Jihyeon Lee, Sang Ho Yoon, Jeongmi Lee

cs.HC ·submitted 2026-09-03 10:59 UTC · v1
published

Abstract

As XR matures into a ubiquitous computing platform, the disconnect between 2D and 3D input modalities remains a critical barrier to seamless workflow. Frequent transitions between the mouse for 2D precision and hand gestures for 3D manipulation induce significant physical fatigue and cognitive load. To address this, we introduce PlanePivoting, a multimodal interaction technique that extends standard mouse input into 3D space by leveraging gaze-mouse alignment. This technique dynamically modulates the translation plane based on the spatial overlap between the gaze and mouse cursor, eliminating the need for physical input modality switching. To systematically explore the foundational design space of gaze-mouse coordination and optimize key variables, we conducted a user study comparing PlanePivoting with a standard 3D Gizmo interface across two translation mapping profiles and two gaze cursor apertures. Results demonstrate that PlanePivoting outperforms the Gizmo on efficiency metrics while maintaining comparable precision and yielding higher subjective satisfaction. This study demonstrates the potential of gaze-mouse alignment for efficient spatial manipulation between 2D and 3D environments.

Comment
8 pages, 5 figures, Accepted to ISMAR'26 Workshop - GEMINI

Closest papers

Nearest neighbours by meaning across the whole index.

UniScale: Exploring Unimanual Gesture Mapping Strategies for Gaze+Pinch-based Scaling Interaction

cs.HC ·2026-09-01 ·Kyoungwhan Mheen, Jinwook Kim, Sang Ho Yoon
published similarity 0.72
Abstract · 11 pages, 7 Figures, Accepted to ISMAR'26, TVCG'26

Object scaling serves as a fundamental spatial manipulation that enables complex and productive tasks in XR environments. This paper investigates unimanual scaling techniques for XR using gaze and hand interactions. We propose UniScale, a set of unimanual alternatives to the standard bimanual pinch, allowing users to scale objects while preserving hand availability for concurrent spatial manipulations. We design five distinct mapping strategies based on physical metaphors, exploring unimanual control that varies depth, angle, micro-gestures, and finger-distance input. We then compare these techniques against a standard bimanual baseline, in which users adjust the inter-hand distance via a bimanual pinch gesture. In a user study, we evaluate their effectiveness in a 3D object scaling task under both clutching and clutching-free conditions. The results indicate that while bimanual scaling relies on clutching for stable control, unimanual techniques excel in clutching-free conditions, significantly reducing physical hand movement. From the results, we derive valuable design implications for developing efficient 3D multimodal interactions in XR.

0.34

GazeTune: Facilitating Precise Gaze-Driven Interactions with Cascaded Touch Input

cs.HC ·2026-09-01 ·Jina Kim, Eric J. Gonzalez, Yang Zhang, Sang Ho Yoon
top venuehot direction similarity 0.70
Abstract · 13 pages, 8 Figures, Accepted to UIST'26

Eye gaze has become an essential input for spatial computing, but its coarse targeting and saccadic nature limit precision and complicate continuous interactions such as dragging, especially under user motion. Gaze+pinch has also become standard in XR for its convenience, yet mid-air gestures remain imprecise, fatiguing, and socially unacceptable. These limitations underscore the need for an approach that preserves the speed of gaze while enabling stable, fine control. We present GazeTune, a cascaded multimodal interaction technique combining gaze and touch to refine gaze-based selection and manipulation. Touch serves as a refinement channel within gaze pointing, allowing precise cursor and target control. Our work investigates how gaze-and-touch enhances dragging and mitigates Motion-Induced instability. In a study (N=20), we compared GazeTune against gaze-only and gaze-pinch methods in 2D dragging. Results show that GazeTune achieves significantly lower error with comparable execution time, validating its effectiveness and balanced trade-off between time and accuracy.

0.47

GRAZE: Text-Free Grasp Synthesis for Unseen Objects from Human Gaze

cs.CV ·2026-11-16 ·Jeongho Lee, Boeun Kim, Hyung Jin Chang, Jungchan Cho, Sang-Il Choi
benchmark similarity 0.65
Abstract · 23 pages, 12 figures

Text-conditioned hand-object interaction (HOI) generation provides intuitive semantic control, but language is an indirect way to specify where an interaction should occur on an object. Naming the part to grasp requires object-specific words, and naming the approach side requires a reference frame that a prompt lacks. We propose GRAZE (GRAsp from gaZE), which generates hand-object grasp motion from recorded gaze instead of object and part names. Its Geometry-Aware Gaze Encoding (GeoGaze) maps gaze rays to cues on object geometry, and cross-attention conditions a diffusion Transformer on these cues. We also introduce HOT3D-Grasp, a grasp-centered benchmark with object-disjoint folds built on HOT3D recordings. On unseen objects, GRAZE grasps the target part more often than the strongest text-conditioned baseline (49.3% vs. 43.6%), achieves the highest grasp success (66.3% vs. 46.6%), and transfers without fine-tuning to GazeHOI (45.3% vs. 33.8%). Gaze conveys the approach side: when gaze is rotated around the object, the approach turns with it in 59.1% of cases, against 20.5% for a text model retrained with clock words. On the same backbone, a text prompt names parts better, but gaze adds the approach side. Given both, GRAZE keeps the prompt's part accuracy and follows gaze.

0.30

Beyond Object Selection:Markerless Gaze-based Robot Placement at Arbitrary Positions

cs.RO ·2026-08-31 ·Yuzhi Lai, William Marx, Shenghai Yuan, Peizheng Li, Zhuoyu Ran +1 more
benchmarkhot direction similarity 0.65
Abstract

Gaze-based assistive manipulation typically supports object selection, while arbitrary-position placement requires accurate spatial alignment between the headset and robot. However, for gaze-based manipulation, pose accuracy does not necessarily translate into task accuracy: translational and rotational errors jointly affect the transformed gaze ray and may compensate for each other. To study cross-device alignment from this task-oriented perspective, we present a markerless interaction framework and a dedicated cross-device dataset. We propose Graph-based Reference Selection to address sparse robot references. We further develop and benchmark multiple task-specific alignment pipelines under a unified protocol. Specifically, we introduce Gaze--Surface Intersection Error (GSIE), which directly measures the spatial error of the gaze-specified target. Experiments show that alignment methods ranked highly by conventional pose metrics are not always optimal in GSIE, demonstrating the importance of evaluating gaze-based manipulation at the task level.

0.30

Exploring Avatar-Based Representations of Desktop Analytical Workflows for Asymmetric Collaborative Visual Analytics

cs.HC ·2026-09-14 ·Tiansu Chen, Yalong Yang, Wai Tong
similarity 0.65
Abstract

Collaborative visual analytics increasingly occurs across asymmetric desktop-VR settings, with desktop analysis offering precision and efficiency and VR providing spatial and embodied affordances. However, maintaining workspace awareness remains challenging because desktop collaborators are often represented in VR only through indirect cues such as perspective sharing, shared visualization state, or lightweight cursor traces, which do not convey their ongoing analytical activity in a VR-native manner. To address this gap, we present Desk2Avatar, which explores avatar-based representations of desktop analytical workflows in VR. We introduce two representation strategies, Distanced and Embodied, inspired by remote pointing and direct manipulation. We conducted a within-subject study with 18 participants comparing these strategies against a depth-adaptive cursor baseline. Our findings show that avatar-based representations improved collaborator awareness and attentional guidance over cursor cues, while excessive embodiment introduced occlusion, distraction, and additional workload. Finally, we discuss design implications for future desktop-to-VR representations, focusing on balancing collaborator presence, attentional guidance, and workspace readability.

0.24

Feel My Focus: Gaze-Triggered Digital Twin Creation for Remote Collaboration

cs.HC cs.CVcs.GR ·2026-11-08 ·Shih-Yu Ma, Suibi Che-Chuan Weng, Ming-Yuan Pai, Tom Yeh, Ellen Yi-Luen Do
top venue similarity 0.59
Abstract · 5 pages, 3 figures

We present Feel My Focus, a gaze-triggered mixed reality (MR) demonstration that enables rapid creation and sharing of lightweight 3D digital twins during remote collaboration. Unlike systems that use gaze primarily as a referential pointer, Feel My Focus treats sustained visual attention as a generative action: a fixation on a physical object initiates single-image capture, attention-guided segmentation, and one-shot 3D reconstruction, allowing a remote partner to inspect and discuss the object in situ. The system integrates gaze-tracking glasses, modular segmentation and reconstruction services, and an MR client for runtime streaming and interaction. This design prioritizes responsiveness and experiential continuity over photometric fidelity. We report observations from a public demonstration, highlighting design trade-offs, failure modes, and user control considerations for deploying gaze-triggered generative systems in real-world collaborative settings.

0.32

Point&Spawn: Mid-Air Reference-Free Object Instantiation Using Gaze and Hand Gestures in Extended Reality

cs.HC ·2026-09-03 ·Jihyeon Lee, Ken Pfeuffer, Jinwook Kim, Jeongmi Lee
top venue similarity 0.59
Abstract · 11 pages, 6 figures, Accepted to ISMAR'26

Mid-air object instantiation in XR requires users to specify a 3D position without spatial references, such as surfaces or existing objects. We present Point&Spawn, a staged pipeline for pre-instantiation position specification through Direction Setting, Depth Setting, and Position Refinement within a continuous gesture flow. We evaluated six controller-free techniques combining Gaze or Non-Dominant Hand (NDH) direction setting with Ray Intersection, Relative Gain, or Drag&Hold depth setting in a user study (N=24) across Near and Far spawn depths. Relative Gain and Drag&Hold yielded faster and more accurate spawning, lower workload, higher usability, and greater preference than Ray Intersection. The shoulder-referenced NDH ray improved speed and coarse accuracy, whereas the viewpoint-based Gaze ray reduced hand movement with comparable final accuracy. Farther spawn depth imposed greater temporal costs as well as Gaze and accuracy costs with Ray Intersection. These findings offer empirical guidance for designing direction and depth control in spawning in XR.

0.44

Evaluating Tactile Encoding Strategies for Communicating 3-D Shape

cs.HC ·2026-11-10 ·MacKenzie C. Harnett, Rebecca F. Friesen
similarity 0.58
Abstract

Two-dimensional haptic displays, such as programmable pin arrays and vibrotactile or friction-modulating touchscreens, enable tactile communication of complex shapes in a low-profile form factor. However, many shapes of interest, including CAD models, three-dimensional graphs, and map topographies, are inherently three-dimensional. This work investigates two methods for encoding 3-D information using a 2-D tactile display: (1) a single tactile height map providing coarse but quantifiable depth cues, and (2) a series of binary layers communicating sequential slices of a 3-D model, supporting more complex geometries such as overhangs and cavities. We further compare performance across two display types--a friction-modulating display and a pin array--to examine how the level of haptic abstraction affects users' ability to decode discrete 3-D information. A user study demonstrates that participants were able to effectively complete a 3-D assembly task using both strategies, achieving high accuracy across all display types. While the pin array supported faster task completion and higher-rated--though not statistically significant--user experience than the friction-modulating display for the binary-layer task, these differences did not translate to significant differences in overall usability (i.e., reassembly accuracy success). Together, these results identify design trade-offs and inform development of future surface haptic displays that support the communication and interpretation of discrete 3-D information in tactile graphics.

0.12