CS Paper Radar

Comparing Hand and Controller Avatars with Hand Tracking and Controller-Based Interaction

Natalia Ocampo, J. Felipe Gonzalez, Robert J. Teather

cs.HC ·submitted 2026-09-18 14:13 UTC · v1
top venuehighly cited

Abstract

Previous research suggests that the congruency between common VR input devices - such as controllers or hand tracking - and their visual representations (e.g., hand or controller avatars) influences user experience and performance. However, the specific effects of input-avatar combinations remain underexplored. We study the effects of common input devices (hand tracking and controllers) and visual representations (hand and controller avatars) on performance and perceived success in target acquisition tasks. We included both grasping and pinching gestures across 16 combinations of input, avatar, and target size. Results indicate that hand tracking benefits from any form of visual representation - even when mismatched - achieving up to 5.8% greater accuracy compared to having no avatar, likely due to its reliance on visual feedback in the absence of a physical prop. Controllers were generally preferred and offered faster task completion. However, mismatched avatars had a stronger negative effect with controllers, particularly when the virtual gesture did not align with the physical action, leading to a 5.6% drop in accuracy compared to the matched condition - suggesting that inaccurate feedback can be more disruptive than having no avatar feedback at all.

Journal ref
2025 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
DOI
10.1109/ISMAR67309.2025.00029

Closest papers

Nearest neighbours by meaning across the whole index.

Comparing Haptic Feedback Across Hand Tracking and Controllers in VR Object Interaction Tasks

cs.HC ·2026-09-18 ·Natalia Ocampo, J. Felipe Gonzalez, Robert J. Teather, Kiyoshi Kiyokawa
published similarity 0.79
Abstract

Hand tracking offers natural VR interaction but lacks controllers' inherent tactile feedback, while haptic feedback across input methods remains understudied. We conducted a participant study comparing vibration, impulse, and no feedback in hand- and controller-based VR interaction across grasp-, pinch-, and tap-like gestures assessing performance, workload, and preference. A custom glove and controller-mounted device provided both feedback types, aiming for consistency across input modalities. Controllers were faster for grasp and pinch, whereas hand tracking was more accurate for pinch and preferred overall. Haptics had limited performance effects, although impulse reduced grasp accuracy with controllers. Participants preferred vibration and impulse over no haptic feedback, favouring vibration overall. Our findings reveal a more nuanced relationship between haptic feedback, input device, and interaction context than suggested by previous work.

0.36

Selfrionette: A Fingertip Force-Input Controller for Continuous Full-Body Avatar Manipulation and Diverse Haptic Interactions

cs.HC ·2026-11-02 ·Takeru Hashimoto, Yutaro Hirao
top venuehighly cited similarity 0.77
Abstract · 14 pages, 12 figures. Author-prepared version of the UIST 2024 paper. Both authors contributed equally

We propose Selfrionette, a controller that uses fingertip force input to drive avatar movements in virtual reality (VR). This system enables users to interact with virtual objects and walk in VR using only fingertip force, overcoming physical and spatial constraints. Additionally, by fixing users' fingers, it provides users with counterforces equivalent to the applied force, allowing for diverse and wide dynamic range haptic feedback by adjusting the relationship between force input and virtual movement. To evaluate the effectiveness of the proposed method, this paper focuses on hand interaction as a first step. In User Study 1, we measured usability and embodiment during reaching tasks under Selfrionette, body tracking, and finger tracking conditions. In User Study 2, we investigated whether users could perceive haptic properties such as weight, friction, and compliance under the same conditions as User Study 1. Selfrionette was found to be comparable to body tracking in realism of haptic interaction, enabling embodied avatar experiences even in limited spatial conditions.

0.61

Beyond the Mirror: Balancing Interaction Modality and Avatar Fidelity in Public 3D Virtual Try-On Systems

cs.HC ·2026-08-24 ·Yueqian Guo, Tianzhao Li, Xin Lv
similarity 0.71
Abstract

Virtual Try-On (VTON) systems deployed on large public displays face a dual barrier: the physical strain of mid-air interaction and the social inhibition caused by public self-consciousness. This paper presents a real-time 3D avatar system integrating markerless motion capture with dynamic visual fidelity control to investigate and mitigate both barriers. Through a dual-study empirical evaluation, we first decoupled physical fatigue from gesture interaction ($N=20$), demonstrating that interaction fatigue is primarily driven by visuomotor latency rather than the physical act of gesturing; our optimized low-latency gesture pipeline achieved usability comparable to touchscreens while delivering superior immersion and hygiene. Building on these insights, our second study ($N=25$) investigated the "avatar fidelity paradox" via a $2 \times 2$ factorial design manipulating interaction modality (gestures vs. touch) and visual fidelity (photorealistic MetaHuman vs. stylized mannequin). Results reveal that while high fidelity and mid-air gestures independently maximize virtual embodiment ($p < .05$), their combination elicits the highest social awkwardness. Crucially, low-fidelity avatars serve as a "psychological mask" that alleviates public embarrassment during expressive gestures, while mid-air gestures simultaneously act as a compensatory mechanism to preserve perceived try-on trust despite reduced visual realism. Finally, we propose a context-aware fidelity framework to balance privacy, immersion, and commercial trust in public spatial interactions.

0.08

Altered Visual Hand Feedback in Virtual Reality: An Exploratory Study in Essential Tremor

cs.HC cs.GR ·2026-10-30 ·Alejandro Pascual-Valdunciel, Marcos Sacristán-Moraleda, Verónica Giménez de Béjar, Julián Benito-León, Juan Marín-Lahoz +2 more
similarity 0.65
Abstract · 12 pages, 5 figures, 3 tables. Under revision

Pathological tremor is the most prevalent movement disorder worldwide, characterized by involuntary muscular oscillations within a frequency of 3 to 12 Hz. Despite extensive research, the underlying mechanisms that generate and modulate tremor are not yet understood, and treatments are usually insufficient. While it is known that visual feedback is an important afferent pathway, little is known about the impact of altering the hand's visual representation on tremor expression. With the development of immersive technologies, VR poses an excellent tool to study how alterations to the representation of the hands affect tremor. In this work, we study this through two complementary experiments. In the first one, we show that a consumer-level hand-tracking system substantially attenuates tremor-band movements without preventing successful task completion. In the second one, with eight patients with essential tremor, we observe task-dependent differences in physical tremor between VR and the real world, with several patients exhibiting lower tremor-band activity in both hand kinematics and muscle oscillations, particularly during reaching tasks. Our findings represent an initial step toward validating the use of VR as an experimental tool for the research and treatment of pathological tremor, while highlighting strong user- and task-dependent effects.

0.19

TapID: Rapid touch interaction in virtual reality using wearable sensing

cs.HC eess.SP ·2026-10-19 ·Manuel Meier, Paul Streli, Andreas Fender, Christian Holz
top venuepositionhighly cited similarity 0.65
Abstract · accepted to IEEE VR 2021

Current Virtual Reality systems typically use cameras to capture user input from controllers or free-hand mid-air interaction. In this paper, we argue that this is a key impediment to productivity scenarios in VR, which require continued interaction over prolonged periods of time---a requirement that controller or free-hand input in mid-air does not satisfy. To address this challenge, we bring rapid touch interaction on surfaces to Virtual Reality---the input modality that users have grown used to on phones and tablets for continued use. We present TapID, a wrist-based inertial sensing system that complements headset-tracked hand poses to trigger input in VR. TapID embeds a pair of inertial sensors in a flexible strap, one at either side of the wrist; from the combination of registered signals, TapID reliably detects surface touch events and, more importantly, identifies the finger used for touch. We evaluated TapID in a series of userstudies on event-detection accuracy (F1 = 0.997) and hand-agnostic finger-identification accuracy (within-user: F1 = 0.93; across users: F1 = 0.91 after 10 refinement taps and F1 = 0.87 without refinement) in a seated table scenario. We conclude with a series of applications that complement hand tracking with touch input and that are uniquely enabled by TapID, including UI control, rapid keyboard typing and piano playing, as well as surface gestures.

0.63

AdaptType: Accessible and Customizable Multimodal Text Entry in Virtual Reality for People with Upper-Body Motor Impairments

cs.HC ·2026-11-14 ·Junhan Kong, Mingyuan Zhong, Sanjana Pavani, Jacob O. Wobbrock
similarity 0.65
Abstract

Text entry in virtual reality (VR) is cumbersome, especially for people with upper-body motor impairments. Existing techniques often demand both fine and gross motor control, yet users' motor abilities vary widely. To address this range of needs, we present AdaptType, a multimodal VR text entry system that enables users to configure input techniques that best suit their abilities, and scaffolds customization through ability-based recommendations. Through a study with 10 participants, we found that customized multimodal techniques improved text entry speed compared to gaze- or hand-only techniques, and complemented voice for error correction, decreasing failure rate from 15.1% to 2.0%. We found that AdaptType yielded high text entry speed (36.26 WPM) and accuracy (0.36% error rate), with low task load and high usability ratings. We also found ability-based recommendations effectively scaffolded customization. Our work contributes more accessible and personalized VR text entry solutions for people with upper-body motor impairments.

0.24

Beyond Task Performance: Lessons Learned from Evaluating an Exploratory VR Interaction Technique

cs.HC ·2026-09-21 ·Nevzat Umut Demirseren, Corey Pittman, Isayas Berhe Adhanom, Karthikeyan Umapathy, Kevin Pfeil
similarity 0.65
Abstract · 6 pages, 10 figures

Dense virtual environments present significant challenges for object selection and manipulation, motivating the development of novel interaction techniques. This paper presents RodCast as an exploratory case study to investigate explicit trajectory visualization for interaction in dense virtual environments. We conducted a within-subjects user study comparing RodCast with Go-Go Hand and FlowerCone across three representative interaction tasks using objective performance measures, subjective evaluations, and qualitative feedback. The results revealed that the proposed implementation incurred performance costs on more demanding manipulation tasks, while qualitative feedback suggested that participants attributed these challenges to the complexity of the bimanual control scheme. Additionally, subjective evaluations and qualitative feedback identified benefits in spatial awareness and target accessibility that were not fully reflected by conventional performance measures. Together, these findings highlight the importance of control simplicity in interaction technique design and emphasize that incorporating qualitative feedback into the evaluation process is essential for distinguishing implementation limitations from the potential of the interaction concept. Collectively, these lessons provide guidance for the design and evaluation of future interaction techniques.

0.26

Externalizing Hand Grip Pressure Through Real-Time Visual Feedback

cs.HC ·2026-10-27 ·Andrew L. Huard, Avinash Ajit Nargund, Misha Sra
hot direction similarity 0.64
Abstract

Grip pressure, including contact force and its distribution across the hand, is central to skilled grasping but is not directly visible. This makes it difficult to teach, learn, and self-correct in grip-dependent motor tasks, where verbal instruction and tactile demonstration provide only partial guidance and no real-time, spatially resolved feedback tied to the learner's own grip. Using an instrumented glove, we compare pressure feedback visualizations for such guidance, evaluating how visualization type and, secondarily, sensor-region mapping shape pressure-matching performance in novices. In a study using baseball and table tennis as representative grip tasks (N=28), we find that a continuous gradient visualization produces significantly more accurate pressure matches and lower gaze entropy than a discrete bar display, despite participants perceiving the bar as more precise for pressure magnitude. Perceived precision and objective performance therefore diverged: the encoding participants judged more precise was not the one they matched more accurately with. Grip-tailored sensor-region mappings further reduced perceived temporal demand and effort, although this subjective workload advantage did not extend to objective matching accuracy. Overall, continuous pressure feedback supported more accurate grip matching and more concentrated viewing, and we present design considerations for real-time pressure feedback in motor skill guidance.

0.26