CS Paper Radar

Where do we Jam? - The Impact of Real and Virtual XR Spaces on Musical Creativity, Co-Presence, Affect, and Flow

Ruben Schlagowski, Jonas Bordewick, Luca Turchet, Hannes Ritschel, Elisabeth André

cs.HC ·submitted 2026-11-10 12:59 UTC · v1

Abstract

Extended Reality (XR) technologies can place musicians in virtual, mixed, or real environments during networked music performance (NMP). To examine how NMP applications in XR should be positioned on the reality-virtuality continuum, we conducted a laboratory study (N = 28) comparing a simulated NMP experience in Virtual Reality (VR) with one in Augmented Reality (AR). In both conditions, musicians saw their remote partners in real time as 3D point clouds rendered with an optimized Gaussian Splatting shader. While VR allowed participants to select and customize fully virtual 3D environments, AR showed a pass-through view of the laboratory environment. We measured creativity support, flow, involvement, co-presence, and affect, and complemented these data with qualitative interviews on participants' preferences and requirements for creative environments. Our results show that participants felt significantly more involved by the immersive XR experience in VR than in AR. Positive affect, flow, and creativity support were strongly correlated in both conditions, whereas involvement correlated with flow and creativity support only in VR. Qualitative findings showed substantial variation in participants' preferences for AR or VR and also regarding the design of creative virtual environments. Together, these findings suggest that the surrounding environment is a critical design dimension in networked Musical XR, shaping collaborative musical experiences beyond merely representing remote collaborators and mitigating communication latency. Future XR-mediated NMP systems should therefore treat the surrounding space as an integral component of the collaborative musical experience, rather than as a mere visual backdrop. At the same time, they should support experiences across a broad spectrum of the reality-virtuality continuum to accommodate highly individual user requirements.

Comment
Accepted at IEEE International Symposium on the Internet of Sound (IS2) 2026, 3rd International Workshop on the Musical Metaverse (IWMM)

Closest papers

Nearest neighbours by meaning across the whole index.

CrowdForge: Effects of Audio and Visual Audience Participation Modalities on Agency, Collaboration, and Engagement in AI-Mediated MR Performances

cs.HC cs.AIcs.NIcs.PF+1 ·2026-11-02 ·Torin Hopkins, Shih-Yu Ma, Zander Trafton-Hancock, Ellen Yi-Luen Do, Luca Turchet
position similarity 0.62
Abstract

Technology-mediated audience participation (TMAP) enables audiences to influence live performances, yet balancing interaction with meaningful agency remains challenging. While AI-generated music and MR enable new forms of audio and visual audience participation, little is known about how these participation modalities influence audience experiences during live musical performance. In particular, it remains unclear how they shape perceived agency, collaboration, and engagement, and how they complement one another within TMAP systems. We present CrowdForge, an AI- and MR-mediated paradigm where audiences collectively control generative music and virtual environments during a live performance featuring a guitarist and DJ. Through a mixedmethods concert study evaluating three interaction conditions (audio, visual, and combined audio-visual), we demonstrate that observability is the primary driver of audience agency. While visual interaction provided easily observable feedback, audio parameters offered macro-structural control over the music. Audiences tended to use the control enabled by AI audio generation to test musical boundaries, introducing a productive adversarial tension between the crowd and the musicians. This dynamic pushed creative limits, deepened engagement, and generated moments of musical coherence when performers successfully resolved audience-induced challenges. We argue future TMAP systems should prioritize human-centered, highly observable interactions to build agency, while combining the expressiveness of music with the clarity of visual feedback.

0.19

Human-AI Co-Ideation Across Socio-Technical Configurations: An Interactional Analysis of Planar and Immersive Workspaces

cs.HC cs.AI ·2026-11-16 ·Orit Shaer, Mary Calabro, Zoe Mitchell, Angelora Cooper, Raffaella Sadun +2 more
similarity 0.61
Abstract

Remote collaborative platforms increasingly integrate generative AI to support group ideation. These platforms are sociotechnical configurations in which spatial mediation, interaction modality, AI output format, and the mechanisms that make contributions visible to a group co-vary as properties of a deployed system. This paper examines how such configurations shape participants' collaboration and their engagement with AI during creative work. We report a comparative study of two commercially available platforms that instantiate contrasting configurations: a planar shared canvas with text-based AI integration, and an immersive virtual reality workspace with concise AI output embedded in the shared space. Fourteen all-women triads completed a structured ideation task consisting of human-only idea generation, AI-supported idea expansion, and group convergence. We analyzed collaboration patterns, selection of AI-generated ideas, perceived idea quality, workload, presence, and their lexical characteristics. The immersive configuration yielded greater spatial presence and introduced a participation paradox, in which spatial freedom enabled exploration and fragmented group coherence; it was also associated with more balanced role distribution during convergence, at the cost of higher physical demand. In both configurations, AI multiplied the idea pool, and ideas groups authored with AI available were rated above participants' own. AI output format: about 21 words per idea on the planar canvas and 3 in the immersive workspace. Tracked how participants evaluated AI-generated ideas and how often they selected them unaltered as finalists. Neither configuration made AI contributions visible or attributable to the group: participants invoked AI individually and asynchronously in both. We derive design implications for AI output format, group-visible AI invocation, and spatial support for convergence

0.09

Auditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic Auditory Distance Perception

eess.AS ·2026-11-05 ·Johannes M. Arend, Christian Scheer, Melissa Ramírez, Stefan Weinzierl, Fabian Brinkmann
similarity 0.57
Abstract · Submitted to a peer-reviewed journal

Extended reality (XR) enables controlled yet naturalistic multisensory experiments, but its validity depends on whether virtualized cues are perceptually plausible and lead to behavior comparable to the real world. To the best of our knowledge, this is the first study to perceptually evaluate a six-degree-of-freedom (6DoF) XR system that combines real-time room acoustic simulation, binaural rendering, and an audio-visually matched virtual replica of a large laboratory room in terms of both plausibility and behavior. In two experiments with 45 participants, we investigated how auditory and visual virtualization affect plausibility and dynamic auditory distance perception (ADP). A plausibility experiment compared real and virtual sound sources in audio-augmented reality (AAR) and virtual reality (VR), while a locomotion-based ADP experiment assessed distance-judgment accuracy, signed error, and task effort across four virtualization conditions. The system produced plausible binaural rendering in both AAR and VR, with higher plausibility in VR, indicating that perceptual requirements for XR audio depend on the visual reference frame. In the ADP task, any form of virtualization reduced accuracy and increased effort, although performance remained high overall. Virtual audio produced a bias toward nearer perceived source distances, whereas visual-only virtualization reduced accuracy without a significant directional bias. These effects did not add up in full VR. Overall, the findings demonstrate the strong potential of current 6DoF XR systems for interactive perceptual research, while revealing that task-specific behavioral validation may be necessary.

0.31

Audiovisual Disparities in VR: Impact on Spatial Perception

cs.GR ·2026-11-06 ·Edurne Bernal-Berdun, Mateo Vallejo, Qi Sun, Ana Serrano, Diego Gutierrez
top venue similarity 0.56
Abstract · 2025 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

Virtual reality (VR) experiences often leverage rich and spatialized multimodal environments to increase immersion and engagement. This demands a consistent spatial perception of audiovisual stimuli, since perceived discrepancies can disrupt the sense of presence. In this work, we investigate the consequences of two types of spatial audiovisual disparities: true disparity, where there is a measurable spatial offset between auditory and visual cues, and perceptual disparity, where users report misalignment despite cues being colocated. Unlike most previous studies that employed controlled but simplified experimental setups, our research focuses on complex, realistic VR environments, allowing us to assess the actual implications for VR content design. Our experiments indicate that users are highly sensitive to true audiovisual disparities in controlled environments, detecting even minor misalignments. However, when engaged in additional tasks within realistic settings, their ability to notice such discrepancies diminishes significantly. We also observed that previously found perceptual disparities persist in complex audiovisual environments. However, we identify self-initiated head rotations as a key factor; its absence prevents the effect entirely. We hope our findings offer practical insights for designing more immersive and perceptually coherent VR experiences.

0.58

AdapTest: Supporting Designers in Authoring and Evaluating Environmentally Responsive User Interfaces for Extended Reality

cs.HC ·2026-10-15 ·Jungwoo Rhee, Guhn Lee, Yumin Jung, Mustafa Doga Dogan, Kongpyung +2 more
published similarity 0.55
Abstract · Accepted to PACM HCI ISS 2026

Extended Reality (XR) headsets increasingly host familiar 2D user interfaces as windows floating in physical environments. However, their usability depends on environmental conditions that designers cannot fully control. Current design workflows often separate how a variant is designed from how it actually looks in the target environment. We present AdapTest, an in-situ authoring and testing tool that lets designers select interface elements, define when and how they change, compare the resulting variations, and test their activation without coding. Based on prior research and a formative study with 12 interface designers, the system targets three key environmental variables: distance, lighting, and occlusion. We evaluated AdapTest against a desktop design and in-headset preview workflow in a within-subjects study with 12 designers. Compared with the baseline, designers accounted for a wider range of environmental factors, addressed more overlapping conditions, and repeatedly tested and revised their designs in place; there were no significant differences in workload or usability despite longer sessions. Our work contributes an integrated, designer-centered approach to adapting, comparing, and validating environmentally responsive XR user interfaces.

0.30

Encypher: Shared Agency and Social Presence in Collaborative Music Generation for Dance Cyphers

cs.HC cs.SDeess.AS ·2026-09-16 ·Zhixing Chen, Cheng-Zhi Anna Huang
code/data similarity 0.55
Abstract · Preprint. Under review at CHI 2027. 11 figures, 1 table. Project page: https://encypher-chi.github.io/

Music and dance are social practices of expression and connection, yet most HCI work in human-AI co-creation centers the solo performer. As generative music matures, we ask not only what AI can compose but what social encounters it can organize around sound. We present Encypher, a collaborative generative music system that translates collective movement qualities into text prompts conditioning real-time music generation for dance cyphers. Through five weeks of co-design with local dancers, a user study with unacquainted participants, a public museum event, and a live performance, we found that users developed shared agency, perceiving the music as a response to the room's energy. While newcomers felt uncertain, the system fostered social presence by prompting them to look to each other for cues. By treating sociality as a design concern rather than a downstream effect, we offer a framework and design implications for AI systems for collaborative, embodied expression.

0.24

Exploring spatial configuration and group familiarity when co-watching 360-degree stories in social virtual reality

cs.HC ·2026-10-21 ·Portia Wang, Monique Santoso, Yujie Tao, Jeremy Bailenson
similarity 0.54
Abstract

While traditional cinematic viewing is often characterized as a social experience, engaging with immersive stories is predominantly researched as an individual activity. This mismatch becomes increasingly problematic in social virtual reality (VR), where features designed for typical social activities (e.g., virtual locomotion, spatial and social context) can become meaningful levers for shaping the user experience. To address our limited understanding of these effects, we conducted three exploratory experiments (N$_1$=109, N$_2$=116, N$_3$=88) examining how spatial boundary type, size of allocated personal space, and group familiarity influence co-watching 360-degree stories in social VR. Results shows that small communal spatial boundaries heightened social presence towards others compared to individual boundaries, but also increased mental demand. Reducing the size of the personal spatial boundary increased social presence towards others but also increased disorientation and frustration. Finally, participants with higher group familiarity reported stronger attentional focus on the story, as well as higher spatial presence and social presence towards story characters, but also reported greater physical and perceptual demand. We discuss systematic tradeoffs between engagement and cognitive effort, and suggest practical guidelines for designing shared immersive narrative experiences.

0.37

Beyond the 1D Reality-Virtuality Continuum: A 2D Framework for Seven Extended Reality Paradigms

cs.HC ·2026-10-27 ·Rajashekhar Vachiravelu Saminathan
published similarity 0.54
Abstract · Accepted for IndiaHCI 2026 Conference in the Student Research Consortium held at Indian Institute of Technology - Hyderabad, India; 7 pages,…

The Reality-Virtuality (RV) continuum helps in describing the relationship between physical and computer-generated environments. However, the increasing number of Extended Reality (XR) paradigms introduces new technologies. Their characteristics cannot be completely represented by a single real-virtual dimension. It can be noted that XR systems not only vary in their proportion of real and virtual content but also adds, suppresses, or transforms information available to the user. This work proposes a framework for characterizing seven XR realities, namely Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), Augmented Virtuality (AV), Phenomenal Reality (PR), Annexing Reality (AnR), and Diminished Reality (DR). It uses a two-dimensional extension of the conventional 1D RV continuum. The first dimension helps in identification of the position of an XR technology between real and virtual environments. The second dimension captures information addition and subtraction. The proposed approach gives a qualitative basis for comparing existing and emerging XR paradigms within a unified XR framework.

0.25