CS Paper Radar

AdapTest: Supporting Designers in Authoring and Evaluating Environmentally Responsive User Interfaces for Extended Reality

Jungwoo Rhee, Guhn Lee, Yumin Jung, Mustafa Doga Dogan, Kongpyung, Moon, Andrea Bianchi

cs.HC ·submitted 2026-10-15 07:03 UTC · v2
published

Abstract

Extended Reality (XR) headsets increasingly host familiar 2D user interfaces as windows floating in physical environments. However, their usability depends on environmental conditions that designers cannot fully control. Current design workflows often separate how a variant is designed from how it actually looks in the target environment. We present AdapTest, an in-situ authoring and testing tool that lets designers select interface elements, define when and how they change, compare the resulting variations, and test their activation without coding. Based on prior research and a formative study with 12 interface designers, the system targets three key environmental variables: distance, lighting, and occlusion. We evaluated AdapTest against a desktop design and in-headset preview workflow in a within-subjects study with 12 designers. Compared with the baseline, designers accounted for a wider range of environmental factors, addressed more overlapping conditions, and repeatedly tested and revised their designs in place; there were no significant differences in workload or usability despite longer sessions. Our work contributes an integrated, designer-centered approach to adapting, comparing, and validating environmentally responsive XR user interfaces.

Comment
Accepted to PACM HCI ISS 2026
DOI
10.1145/3859325

Closest papers

Nearest neighbours by meaning across the whole index.

MRxD: A Usage-Pattern Framework for Mixed-Reality Cross-Device Interaction

cs.HC ·2026-11-12 ·Yang Zhan, Yuxuan Huang, Yuchen Cai, Yichen Andy Yu, Xiaoran Yang +2 more
similarity 0.64
Abstract · 22 pages, 11 figures

Mixed reality (MR) cross-device experiences often combine MR headsets with non-immersive devices such as laptops, tablets, and phones. Yet such cross-device design remains challenging because users must coordinate workflows across heterogeneous devices with different roles. This motivates our focus on usage patterns: recurrent ways in which people distribute and coordinate work across devices. Through a systematic analysis of 238 MR cross-device papers, we developed MRxD, a usage-pattern framework organized around three dimensions: user attention, task relatedness, and cognitive symmetry. Using these dimensions, we identified six usage patterns and synthesized the design strategies commonly used within each pattern. We further investigated the framework's application in design practice through a workshop (n=6) and the results suggest that MRxD can guide pattern-based design exploration and identify blind spots. We further discuss how usage patterns may shift or combine, and outline research gaps and design implications for future MR cross-device systems.

0.30

Trade-offs in Data Color Palette Design Tools

cs.HC ·2026-08-19 ·Shiyi He, Andrew M McNutt
similarity 0.62
Abstract · 2 pages, IEEE VIS 2026 POSTER

Designing a color palette for data requires designers to balance multiple constraints, including accessibility and aesthetics. Color palette tools support this process through features including direct manipulation, automated palette generation and evaluation, previews, and so on. Despite their prominence, relatively little is known about how these different mechanisms shape design across contexts. We conducted an exploratory think-aloud crowd work study with 40 self-identified designers. Each participant used one of four palette tools selected to span different interaction modalities to complete a series of accessibility- and aesthetics-oriented design tasks. We observed two preliminary patterns. First, tool differences were more pronounced in accessibility-constrained tasks. Second, even when accessibility was not explicitly required, some tools produced more accessibility-friendly palettes and prompted more accessibility-oriented thinking. In this tool genre, then, system design shapes outcomes both via built-in functionality, as well as by directing designers' attention toward particular constraints and design considerations.

0.21

GUIDE: Designer-in-the-loop Authoring of Conformant Generative User Interfaces

cs.HC ·2026-09-18 ·Hyewon Lee, Ziying Wang, Aiden Moy, Saran Nagubandi, Jason Wu
hot direction similarity 0.62
Abstract · 24 pages, 13 figures, 5 tables

Generative User Interfaces (GenUIs) enable applications to generate interfaces on demand from user needs and context. Like conventional UIs, they must still reflect designers' intent and conform to requirements such as brand identity. Unlike conventional UIs, designers cannot directly specify or see every interface a GenUI may produce, making design intent harder to enforce. We introduce GUIDE (GenUI Development Environment), a system that lets designers continuously inspect and refine GenUI behavior as they create and edit interfaces. GUIDE uses designers' modifications and interactions to adapt GenUIs through prompt optimization and a novel adaptive conformance scoring model. We validate GUIDE's scoring model and system. The scoring model matched or outperformed proprietary LLM baselines on held-out comparisons of real and synthetic application screens. In a study with 12 UI/UX practitioners, participants found GUIDE effective and usable and significantly preferred aligned outputs over a strong baseline using exemplars and a model-generated design.md.

0.34

Immersive Commons: Designing Ephemeral Spatial Dialogue for Co-Located Mixed Reality Environments

cs.HC cs.MM ·2026-10-26 ·Wuwu Ran, Mohammed Faraj Al-Suwaidi
published similarity 0.61
Abstract · 14 pages, 8 figures. Accepted at SIGraDi 2026, the Conference of the Iberoamerican Society of Digital Graphics

Head-mounted mixed reality (MR) often prioritises individual immersion, potentially drawing attention away from others sharing the room. This paper asks how MR can support co-located spatial dialogue through embodied interaction and activity visible to non-headset observers. We introduce Immersive Commons, a design framework, and Atmos Messages, a prototype combining gesture-based, ephemeral spatial messages with voice transcription, preset text, and a public floor-plan overview. Following a Research through Design approach, fifteen participants created 236 messages across two sessions, including 48 voice-authored messages. Questionnaire responses highlighted playfulness and social awareness, while observations suggested opportunities for exchange grounded in the architectural setting. Interaction clustered near the public display, although this was also where participants put on their headsets. These findings suggest the potential of ephemeral spatial messaging to support co-located exchange, with playfulness, shared visibility, and architectural context informing the design of more social and situated MR experiences.

0.22

AUV-Bench: Aesthetic Understanding and Generation Evaluation for User Interfaces

cs.AI ·2026-09-28 ·Zhijie Deng, Ling Li, Junhao Ji, Siwei Lyu, Zhipeng Xu +5 more
similarity 0.60
Abstract

Multimodal foundation models are increasingly used for evaluating and generating user interfaces (UIs), often producing seemingly reasonable aesthetic judgments and visually plausible pages. However, under professional design scrutiny, their behavior can differ substantially from that of human designers. In professional design practice, designers rely on a systematic set of aesthetic principles that consistently guide judgment, diagnosis, repair, and creation. A coherent aesthetic capability should therefore connect aesthetic judgment with design actions. Existing evaluations, however, typically assess these abilities in isolation, making it difficult to determine whether task-level success reflects a shared aesthetic understanding or merely fragmented task-specific competence. To address this gap, we introduce AUV-Bench, developed in collaboration with professional UI designers around 1,395 executable web interfaces and four tasks: aesthetic scoring, diagnosis, repair, and text-to-UI generation. The tasks share a pool of UIs and aesthetic principles, with diagnosis and repair further aligned on 660 controlled-degradation instances to enable instance-level analysis of judgment and action. Evaluation of 12 models reveals a capability imbalance: models show moderate agreement with professional designers in holistic aesthetic scoring, yet exact diagnosis-chain success peaks at only 24.7%. On the aligned diagnosis-repair cases, correct judgments and successful repairs do not consistently coincide, exposing a Judgment-Action Gap between identifying aesthetic problems and successfully acting on them. In open-ended generation, even leading models achieve only moderate aesthetic quality under human-calibrated evaluation. Overall, current models exhibit partial aesthetic competence, but still lack the fine-grained understanding and judgment-action coherence required for reliable UI design.

0.13

User Preferences for UI Anchoring in MR: Effects of Task Mobility and Interface Properties

cs.HC ·2026-08-28 ·João Belo, Sina Elahimanesh, Anna Maria Feit
top venue similarity 0.60
Abstract · Accepted to IEEE International Symposium on Mixed and Augmented Reality 2026

Anchoring - the choice of frame of reference for mixed reality (MR) interface elements - is a critical design decision involving trade-offs between accessibility, interaction comfort, and visual interference. Despite its importance, user preferences for anchoring across different mobility contexts and interface properties remain poorly understood, as prior work has largely focused on specific tasks or fixed interface configurations. We address this through a mixed-methods user study in which participants configure anchoring strategies across different mobility conditions and interface types. Combining behavioral analysis with structured qualitative inquiry, we analyze how participants select and reason about anchoring modes. Our results show a clear transition from world-anchored interfaces in stationary contexts to body-anchored interfaces during locomotion. However, no single body anchor consistently dominates, highlighting the personal nature of anchoring strategies. Our qualitative analysis reveals the factors users consider in their anchoring decision, including interface accessibility, stability during interaction, visual clutter, and individual mental models. These findings inform the design of adaptive and controllable MR interfaces and highlight the importance of supporting user customization.

0.53

reVISit-XR: Bringing Extended Reality into Embeddable, Trackable, and Replayable Visualization Studies

cs.HC ·2026-10-06 ·Shano Liang, Max Chen, Lane Harrison
similarity 0.59
Abstract · 5 pages, 2 figures

Extended reality (XR) is increasingly a setting for empirical visualization research, where study-relevant state is distributed across headset and controller pose, scene configuration, selections, spatial layouts, and AR anchors. Existing XR tools support parts of the workflow, such as scene authoring, interaction, or session analysis, yet seldom treat an XR stimulus as a reusable component of a complete study lifecycle. We present reVISit-XR, an extension of reVISit that makes customizable WebXR stimuli embeddable, trackable, and replayable within empirical visualization studies. reVISit-XR sequences XR scenes alongside standard study components, collects reactive task responses, captures scene-authored semantic state together with generic XR traces, and rehydrates participant sessions for later desktop and headset analysis. Organized as a reusable stimulus build package and a study integration package, it lets XR stimuli act as first-class study components. We demonstrate its scope and feasibility through seven integrated, reusable examples and a deployed study, and discuss its current capabilities and future extensions.

0.19

ReRoom: Blending Virtual and Physical Contexts for In Situ Room Planning in Mixed Reality

cs.GR cs.HC ·2026-09-03 ·Hongliang Yang, Yanjing Xu, Anhang Zhang, Hui Ye, Pengfei Xu
similarity 0.59
Abstract · 19 pages, 8 figures; supplementary materials included

Planning a real domestic space is an in situ authoring process: users evaluate candidate layouts at true scale, refine their intent, and carry accepted decisions into later iterations. Existing approaches either separate layout editing from the physical room or provide limited support for evaluating and refining whole-room proposals in situ. We present ReRoom, a mixed-reality system for in situ room-layout authoring. ReRoom presents a shared layout state through a virtual room proxy spatially registered to the target room, allowing interaction and layout generation to remain grounded in the physical context. Users refine the current proposal through direct manipulation or language and preserve accepted placements, allowing each generated update to continue the same evolving design. To balance layout quality with generation efficiency, ReRoom uses a skill-guided layout agent whose room-layout design skill operationalizes three principles that we formulate by synthesizing established interior-design guidance for real-room layout generation. The skill grounds these principles in a normalized representation of the scanned room and reusable geometric checks. Evaluations show that ReRoom produces high-quality layouts for non-rectangular rooms, while its in situ workflow improves the room-planning experience over an otherwise equivalent off-site VR workflow. Code will be released upon acceptance of the paper.

0.36