CS Paper Radar

viaCross: A Pen-Based Technique for Selecting Objects and Attributes of Interest using Crossing-Based Gestures

Benedict Leung, Mariana Shimabukuro, Christopher Collins

cs.HC ·submitted 2026-10-02 19:56 UTC · v1

Abstract

Crossing-based selection is well-studied, yet past research has overlooked multi-object and attribute selection with crossing gestures. We present viaCross, a pen-based technique in which crossing strokes define attribute constraints, paired with attribute widgets that visualize and allow subsequent edits to these constraints. A controlled study compared viaCross with traditional WIMP filter panels. Results show that viaCross was more efficient for complex object-level selections, requiring fewer strokes and maintaining efficiency as task difficulty increased. WIMP panels were faster and more intuitive for simple attribute-based selections, but participants encountered repeated deselections and occasional misclicks. viaCross matched panel performance for creating attribute selections but was slower during refinement, as widgets had to be recreated, indicating the need for persistent widgets or hybrid support with a WIMP-style interface. Overall, viaCross demonstrates that crossing-based gestures can provide an efficient and scalable approach to multi-object and attribute selection.

Closest papers

Nearest neighbours by meaning across the whole index.

GazeTune: Facilitating Precise Gaze-Driven Interactions with Cascaded Touch Input

cs.HC ·2026-09-01 ·Jina Kim, Eric J. Gonzalez, Yang Zhang, Sang Ho Yoon
top venuehot direction similarity 0.59
Abstract · 13 pages, 8 Figures, Accepted to UIST'26

Eye gaze has become an essential input for spatial computing, but its coarse targeting and saccadic nature limit precision and complicate continuous interactions such as dragging, especially under user motion. Gaze+pinch has also become standard in XR for its convenience, yet mid-air gestures remain imprecise, fatiguing, and socially unacceptable. These limitations underscore the need for an approach that preserves the speed of gaze while enabling stable, fine control. We present GazeTune, a cascaded multimodal interaction technique combining gaze and touch to refine gaze-based selection and manipulation. Touch serves as a refinement channel within gaze pointing, allowing precise cursor and target control. Our work investigates how gaze-and-touch enhances dragging and mitigates Motion-Induced instability. In a study (N=20), we compared GazeTune against gaze-only and gaze-pinch methods in 2D dragging. Results show that GazeTune achieves significantly lower error with comparable execution time, validating its effectiveness and balanced trade-off between time and accuracy.

0.47

Magic Pen: Automatic Pen Mode Switching for Document Annotation

cs.HC ·2026-10-08 ·Kevin Desousa, Adam Bradley, Nathalie Henry Riche, Ken Hinckley, Christopher Collins
similarity 0.59
Abstract · Technical report. 18 pages, 18 figures, 3 tables

Traditional digital pen interfaces use menu buttons to change the pen mode, which results in time and cognitive load spent on round-trip interactions and mode errors from tapping small mode selection buttons. This work presents the Magic Pen, a technique which uses machine learning to automatically switch between digital pen modes without requiring explicit mode changes. Magic Pen is driven by an LSTM model trained on pen data collected from 27 participants across two studies and uses transfer learning to iteratively tune the model towards how a specific user annotates. Error mitigation techniques using a flick gesture or on-screen tap are incorporated to correct mode errors or remove a stroke quickly. We evaluated Magic Pen in a comparative study with 18 participants, followed by iterative improvements and a deployment study with 8 participants. Magic Pen was preferred compared to a conventional menu-based approach, and transfer learning allowed for greater model predictability and stability.

0.22

MM-SVGEdit: A Multimodal-Driven SVG Editing for UI Design

cs.HC cs.CV ·2026-09-05 ·Shibo Yang, Yuqing Gao, Zipeng Liu
similarity 0.57
Abstract

In the field of UI design, Scalable Vector Graphics (SVG) is widely used as a design medium. However, traditional SVG editing techniques have high entry barriers and require cumbersome manual iteration, while LLM-based editing solutions suffer from low accuracy and poor user controllability. To address these issues, we propose MM-SVGEdit, a multimodal-driven SVG editing approach that integrates traditional SVG editing and LLM-based methods. We introduce a two-stage strategy in which visual grounding is followed by modification. Both stages support two interaction modalities: natural language instructions and direct manipulation (mouse and keyboard). We trained and evaluated MM-SVGEdit on a self-constructed dataset of 14,476 question-answer pairs generated from UIs, covering 11 types of editing operations on both single and multiple UI targets. The results show that MM-SVGEdit improves SVG editing accuracy, efficiency, and user-perceived control while reducing token consumption and response time.

0.17

Chart the Invisible: Interactive Perception for Hidden-Target Exploration in GUI Agents

cs.SE ·2026-10-26 ·Yongxiang Hu, Ruiyang Liu, Qining He, Xuan Wang, Yangfan Zhou
dataset similarity 0.56
Abstract · 21 pages, 6 figures, and 4 tables

Despite rapid progress in command-line interface (CLI) tasks, recent LLM-based agents still struggle with reliable task execution through graphical user interfaces (GUIs). A key challenge is the limited content visible within a bounded viewport, especially on mobile devices, where many task-relevant targets remain hidden and can only be revealed through swipe interactions. Therefore, although most GUI agents support swipe as an action primitive, the lack of target visibility makes it difficult for them to decide when, where, and how to swipe to reveal the hidden content. Our quantitative study further shows that GUI agents incorrectly choose a click action in 47.8% of cases that require swipe exploration. To bridge this gap, this paper proposes equipping GUI agents with scrollable-region perception to support hidden-area exploration before action selection. To this end, we trained the first multimodal model for scrollable-region perception on a manually collected dataset of 667 UI screens, called Hagasu. By correctly identifying 78.5% of scrollable regions, experiments across nine GUI agents show that integrating Hagasu improves the mean end-to-end completion rate from 64.8% to 82.7% on explore-dependent tasks. To facilitate follow-up research, we release the swipe dataset and the source code of Hagasu.

0.19

WidgetVA: A Widget-Centric Framework and Benchmark for Agentic Visual Analytics

cs.HC cs.AI ·2026-09-21 ·Yutong Chen, Zhike Tang, Zhihao Mai, Zhihao Shuai, Danli Luo +2 more
benchmarkcode/data similarity 0.55
Abstract

Visual analytics (VA) enables sensemaking through interactive visualization, but effective analysis often requires experts to translate high-level intents into long sequences of interface operations and iteratively interpret visual feedback. We study whether modern vision-language models (VLMs) can take on this role as autonomous VA operators that observe the interface, plan multi-step exploration, execute interactions, and adapt based on intermediate visual feedback. To support systematic development and evaluation, we first introduce WidgetVA, a widget-centric agentic VA framework that standardizes interactive components as structured widgets with unified action (e.g., filter and zoom) and perception-query (e.g., selection summaries) APIs. This standardization supports two modes of system construction: wrapping an existing VA system to make it agent-operable without rebuilding it, and composing a new system from widgets as modular building blocks. To help agents coordinate across widgets rather than plan each interaction from scratch, each widget further packages reusable analytical workflows, giving agents more than a bare set of callable functions to plan over. Building on this framework, we present WidgetVABench, a benchmark of single- and multi-widget VA tasks that require agents to perform multi-step interactions to uncover evidence and produce verifiable results. Each task also provides fine-grained reference annotations so that WidgetVABench can score Answer, Reference Trace Similarity, and State separately rather than collapsing agent performance into one success score. Experiments across multiple VLMs show that our framework provides an effective scaffold for agentic VA, while the diagnostic measures expose persistent limitations for future work. The WidgetVA framework and WidgetVABench have been released in https://github.com/Hiverwin/widgetva.

0.38

Present Drawing: Exploring Avatar-Mediated Drawing Events with Physical Traces

cs.HC ·2026-10-23 ·Kojiro Tanaka, Takuya Iwamoto, Yuki Okafuji
top venue similarity 0.54
Abstract · Accepted (ACM UIST 2026)

We introduce Present Drawing, an approach for staging an avatar-mediated drawing event that leaves physical traces in the real world. Present Drawing aims to reconfigure conventional drawing events by relaxing their physical constraints. We describe an organizer-oriented design space for such an event along four elements: Time, Site, Control Source, and Avatar Embodiment. To ground this concept, we developed a proof-of-concept prototype in which an observer wears a head-mounted display to view an avatar. The avatar's motion is synchronized with a customized pen plotter whose mechanical parts are hidden from the observer. We also developed five application examples spanning live, playback, multi-site, mascot, and system-controlled configurations. We conducted an initial evaluation of one representative playback autograph configuration, showing the basic plausibility of the core Present Drawing experience as an avatar-mediated drawing event and highlighting factors associated with perceiving the avatar as the drawer.

0.42

UniScale: Exploring Unimanual Gesture Mapping Strategies for Gaze+Pinch-based Scaling Interaction

cs.HC ·2026-09-01 ·Kyoungwhan Mheen, Jinwook Kim, Sang Ho Yoon
published similarity 0.54
Abstract · 11 pages, 7 Figures, Accepted to ISMAR'26, TVCG'26

Object scaling serves as a fundamental spatial manipulation that enables complex and productive tasks in XR environments. This paper investigates unimanual scaling techniques for XR using gaze and hand interactions. We propose UniScale, a set of unimanual alternatives to the standard bimanual pinch, allowing users to scale objects while preserving hand availability for concurrent spatial manipulations. We design five distinct mapping strategies based on physical metaphors, exploring unimanual control that varies depth, angle, micro-gestures, and finger-distance input. We then compare these techniques against a standard bimanual baseline, in which users adjust the inter-hand distance via a bimanual pinch gesture. In a user study, we evaluate their effectiveness in a 3D object scaling task under both clutching and clutching-free conditions. The results indicate that while bimanual scaling relies on clutching for stable control, unimanual techniques excel in clutching-free conditions, significantly reducing physical hand movement. From the results, we derive valuable design implications for developing efficient 3D multimodal interactions in XR.

0.34

Vibes on Demand: Adding Vibrotactile Encoding to Line Charts Shows Experiential Benefits Without Performance Costs

cs.HC ·2026-08-15 ·Anchit Mishra, Oliver Schneider, Matthew Brehmer
similarity 0.53
Abstract

Details on demand is a common design pattern in visualization design, especially useful when interacting with visually-saturated or small displays. Beyond visualization, another common approach for saturated displays is to incorporate other modalities, such as haptic feedback. While haptic rendering in visualization has primarily targeted accessibility needs, with haptics as a substitute for visual feedback, studies using haptics outside of a visualization context have shown value in experiential factors, such as increased confidence in ambiguous contexts and higher engagement. We explore vibrotactile feedback as a reinforcing information channel for communicating trends in details-on-demand tooltips on touchscreens. We identify preferred parameter configurations for our haptic encoding, informed by a study where participants identified parameter configurations that they perceived to most accurately reflect the dynamics of line charts appearing in tooltips. In a second study, we evaluated participant performance in a pairwise comparison task, finding that incorporating vibrotactile encoding improves involvement without affecting accuracy. We discuss the implications of these findings for future visualization design, and propose directions for applications and future studies.

0.04