CS Paper Radar

Beyond Task Performance: Lessons Learned from Evaluating an Exploratory VR Interaction Technique

Nevzat Umut Demirseren, Corey Pittman, Isayas Berhe Adhanom, Karthikeyan Umapathy, Kevin Pfeil

cs.HC ·submitted 2026-09-21 21:11 UTC · v1

Abstract

Dense virtual environments present significant challenges for object selection and manipulation, motivating the development of novel interaction techniques. This paper presents RodCast as an exploratory case study to investigate explicit trajectory visualization for interaction in dense virtual environments. We conducted a within-subjects user study comparing RodCast with Go-Go Hand and FlowerCone across three representative interaction tasks using objective performance measures, subjective evaluations, and qualitative feedback. The results revealed that the proposed implementation incurred performance costs on more demanding manipulation tasks, while qualitative feedback suggested that participants attributed these challenges to the complexity of the bimanual control scheme. Additionally, subjective evaluations and qualitative feedback identified benefits in spatial awareness and target accessibility that were not fully reflected by conventional performance measures. Together, these findings highlight the importance of control simplicity in interaction technique design and emphasize that incorporating qualitative feedback into the evaluation process is essential for distinguishing implementation limitations from the potential of the interaction concept. Collectively, these lessons provide guidance for the design and evaluation of future interaction techniques.

Comment
6 pages, 10 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Comparing Haptic Feedback Across Hand Tracking and Controllers in VR Object Interaction Tasks

cs.HC ·2026-09-18 ·Natalia Ocampo, J. Felipe Gonzalez, Robert J. Teather, Kiyoshi Kiyokawa
published similarity 0.70
Abstract

Hand tracking offers natural VR interaction but lacks controllers' inherent tactile feedback, while haptic feedback across input methods remains understudied. We conducted a participant study comparing vibration, impulse, and no feedback in hand- and controller-based VR interaction across grasp-, pinch-, and tap-like gestures assessing performance, workload, and preference. A custom glove and controller-mounted device provided both feedback types, aiming for consistency across input modalities. Controllers were faster for grasp and pinch, whereas hand tracking was more accurate for pinch and preferred overall. Haptics had limited performance effects, although impulse reduced grasp accuracy with controllers. Participants preferred vibration and impulse over no haptic feedback, favouring vibration overall. Our findings reveal a more nuanced relationship between haptic feedback, input device, and interaction context than suggested by previous work.

0.36

Exploring Avatar-Based Representations of Desktop Analytical Workflows for Asymmetric Collaborative Visual Analytics

cs.HC ·2026-09-14 ·Tiansu Chen, Yalong Yang, Wai Tong
similarity 0.69
Abstract

Collaborative visual analytics increasingly occurs across asymmetric desktop-VR settings, with desktop analysis offering precision and efficiency and VR providing spatial and embodied affordances. However, maintaining workspace awareness remains challenging because desktop collaborators are often represented in VR only through indirect cues such as perspective sharing, shared visualization state, or lightweight cursor traces, which do not convey their ongoing analytical activity in a VR-native manner. To address this gap, we present Desk2Avatar, which explores avatar-based representations of desktop analytical workflows in VR. We introduce two representation strategies, Distanced and Embodied, inspired by remote pointing and direct manipulation. We conducted a within-subject study with 18 participants comparing these strategies against a depth-adaptive cursor baseline. Our findings show that avatar-based representations improved collaborator awareness and attentional guidance over cursor cues, while excessive embodiment introduced occlusion, distraction, and additional workload. Finally, we discuss design implications for future desktop-to-VR representations, focusing on balancing collaborator presence, attentional guidance, and workspace readability.

0.24

Selfrionette: A Fingertip Force-Input Controller for Continuous Full-Body Avatar Manipulation and Diverse Haptic Interactions

cs.HC ·2026-11-02 ·Takeru Hashimoto, Yutaro Hirao
top venuehighly cited similarity 0.68
Abstract · 14 pages, 12 figures. Author-prepared version of the UIST 2024 paper. Both authors contributed equally

We propose Selfrionette, a controller that uses fingertip force input to drive avatar movements in virtual reality (VR). This system enables users to interact with virtual objects and walk in VR using only fingertip force, overcoming physical and spatial constraints. Additionally, by fixing users' fingers, it provides users with counterforces equivalent to the applied force, allowing for diverse and wide dynamic range haptic feedback by adjusting the relationship between force input and virtual movement. To evaluate the effectiveness of the proposed method, this paper focuses on hand interaction as a first step. In User Study 1, we measured usability and embodiment during reaching tasks under Selfrionette, body tracking, and finger tracking conditions. In User Study 2, we investigated whether users could perceive haptic properties such as weight, friction, and compliance under the same conditions as User Study 1. Selfrionette was found to be comparable to body tracking in realism of haptic interaction, enabling embodied avatar experiences even in limited spatial conditions.

0.61

Interactive Visual Gym: Evaluating Visual Interaction Models with Simulated Users in Interactive Worlds

cs.CV ·2026-11-05 ·Chenhao Zheng, Zixian Ma, Kai Li, Kanchana Ranasinghe, Jieyu Zhang +9 more
benchmark similarity 0.67
Abstract · 39 pages, including references and appendices

Visual interaction models that support users during ongoing physical tasks require evaluation in closed loop, where assistant responses can change subsequent user behavior and task outcomes. Existing benchmarks largely rely on offline replay and therefore cannot capture these effects. We introduce Interactive Visual Gym, a framework for closed-loop evaluation and training of visual interaction models through joint simulation of executable task environments and human users. We contribute three simulators that recreate everyday tasks and human-aligned action spaces across cooking, household manipulation, and mobile-interface domains, together with a persona-conditioned user simulation harness that collaborates with the assistant while incorporating controlled execution errors and other out-of-plan behaviors. We instantiate the framework as IVG-Bench, a benchmark with 315 interaction scenarios. Evaluating state-of-the-art VLMs as visual interaction models reveals substantial limitations: the best-performing model, Claude Opus 5.5, completes only 53.5% of tasks within the time limit, exhibits significant weaknesses in factual grounding and actionable guidance, and is often too slow to prevent errors when response latency is considered. We verify our evaluation protocol with a human study which finds 78.5% agreement between our interaction-quality metric and participant preferences. We further show that the environments provide useful learning signals: in a proof-of-concept GRPO experiment, closed-loop training improves both task success and interaction quality for Qwen3.5-4B model in our cooking environment.

0.27

Wait for It: A Component-Level Analysis of Latency Feedback Mechanisms for LLM Agents in VR

cs.HC ·2026-10-16 ·Pratham Arora, Abhinav Lodha, Siddharth Siddharth
similarity 0.65
Abstract

Delays in AI responses pose a significant challenge to immersive Virtual Reality experiences. While previous research demonstrates that feedback can reduce the perception of waiting, we lack understanding of which types work best and why. We conducted a within-subjects study with 18 participants to systematically compare three isolated feedback approaches during VR agent interactions: verbal fillers (e.g., um..., hmm...), visual indicators (loading spinners), and embodied gestures (thinking animations). Our results revealed that embodied gestures significantly outperformed visual loading indicators across multiple dimensions, making agents feel more lifelike and socially present. Interestingly, explicit visual feedback like loading spinners actually degraded perceived humanness and broke immersion by forcing users to shift from social to technical mental models. These findings suggest that natural, behavior-based feedback maintains the illusion of conversing with a thinking entity, while technical indicators expose the computational substrate. We present empirically-grounded design guidelines for creating more engaging AI-powered VR agents.

0.08

Comparing Hand and Controller Avatars with Hand Tracking and Controller-Based Interaction

cs.HC ·2026-09-18 ·Natalia Ocampo, J. Felipe Gonzalez, Robert J. Teather
top venuehighly cited similarity 0.65
Abstract

Previous research suggests that the congruency between common VR input devices - such as controllers or hand tracking - and their visual representations (e.g., hand or controller avatars) influences user experience and performance. However, the specific effects of input-avatar combinations remain underexplored. We study the effects of common input devices (hand tracking and controllers) and visual representations (hand and controller avatars) on performance and perceived success in target acquisition tasks. We included both grasping and pinching gestures across 16 combinations of input, avatar, and target size. Results indicate that hand tracking benefits from any form of visual representation - even when mismatched - achieving up to 5.8% greater accuracy compared to having no avatar, likely due to its reliance on visual feedback in the absence of a physical prop. Controllers were generally preferred and offered faster task completion. However, mismatched avatars had a stronger negative effect with controllers, particularly when the virtual gesture did not align with the physical action, leading to a 5.6% drop in accuracy compared to the matched condition - suggesting that inaccurate feedback can be more disruptive than having no avatar feedback at all.

0.65

Generative Tutorial: Towards Live Contextualized Visual Instructions for Physical Tasks

cs.HC cs.AI ·2026-09-21 ·Muzhe Wu, Zuchen Li, Xu Wang, Anhong Guo
similarity 0.64
Abstract

Visual instructions for physical tasks are typically authored in one context and followed in another, requiring users to translate demonstrated tools, materials, and spatial relationships into their own environment. We introduce Generative Tutorial, a conceptual framework for live visual instruction that depicts intended outcomes and actions within the user's environment and task flow. A formative evaluation of state-of-the-art image and video generation identifies failures and potential benefits across 15 physical tasks. Drawing on these findings, we build an augmented-reality prototype system that proactively generates goal images and demonstration videos using observed workspace context and predicted visual outcomes of preceding actions. A 24-participant lab study found higher task performance quality, greater perceived workspace correspondence, and shorter step-confirmation intervals with the system than with pre-authored guidance. Qualitative findings highlighted how contextual resemblance shapes trust, how generation errors affect interpretation, and how guidance delivery should adapt to users' needs, informing future designs.

0.41

Tactile Search: Enhancing Targeting in 3D Space

cs.HC ·2026-09-05 ·Amber Maimon, Iddo Yehoshua Wald, Jonas Keppel, Eunhee Chang, Yoshifumi Kitamura +3 more
top venue similarity 0.64
Abstract · Accepted to the 2026 IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2026)

Visual search is crucial in daily life, from scanning for relevant information to spotting signs of danger. When sensory channels are overloaded or degraded, cognitive tasks can be supported by crossmodal information representations through vibrotactile cues. We introduce Tactile Search, an approach that uses modulation of frequency and amplitude of vibrations to the hands, for guiding attention to the location of objects in 3D space. We evaluated this approach in a competitive VR game where participants searched for targets using both vision and touch. Across two studies -- an in-the-wild demonstration (n=55) and a controlled laboratory experiment (n=28) -- we found that vibrotactile feedback significantly improved performance and increased user confidence. In the combined haptic condition, performance did not differ across target heights. We further analyzed participants' subjective experiences and search strategies highlighting the benefits of the tactile cues. Our findings suggest that Tactile Search can enhance interaction and provide design considerations for integrating haptic search into interactive systems.

0.24