CS Paper Radar

How Children Collaborate within Programmable AR Environments with Co-Located Collaborative Features

Romina Mahinpei, Diya Ajay Hundiwala, Sandy Zhang, Lana Glisic, Andrés Monroy-Hernández

cs.HC ·submitted 2026-08-11 21:20 UTC · v1
published

Abstract

Programmable augmented reality (AR) environments are emerging as a promising way to support children's creative learning through embodied interaction with digital characters and physical space. At the same time, AR systems are increasingly capable of supporting co-located collaborative experiences. However, little is known about how children collaborate within programmable AR environments offering co-located collaborative features. In response, we extended Capybara, an existing programmable AR application for children, with co-located collaborative features supporting shared visibility and interaction across devices. We then conducted workshops with 9 children to examine whether and how collaboration emerges during use. Across our workshops, collaboration was often lightweight and implicit, emerging through three complementary forms: parallel play with social awareness, iterative remixing, and spontaneous peer support. Together, our findings provide insights for designing future child-centered programmable AR systems that better support co-located collaborative experiences.

Comment
Published at CSCW 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Context Matters! Social, Spatial, and Semantic Coordination with an Embodied Agent in Immersive Analytics

cs.HC ·2026-10-15 ·Andrew Cunningham, Nicolas Barbotin, Adam Drogemuller, Jack Faser
top venue similarity 0.63
Abstract · In 2025 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) (pp. 189-194). IEEE

Embodied AI agents in Augmented Reality (AR) could enable more natural collaboration in immersive analytics by engaging users through spatial, conversational, and visual interaction. However, designing such agents requires careful attention to context. We propose a preliminary conceptual framework for context-aware design of embodied agents structured around three dimensions of spatial, social, and semantic. This framework is further categorised into embeddedness (how agents are situated in the contexts) and awareness (how agents perceive and respond to the contexts). To explore this, we present Cobot, an embodied AI agent designed as a design probe. Cobot supports context-sensitive interaction through features such as gaze cues, spatial sound, and task-aware behaviour. We reflect on its design and identify implications for future contextaware embodied agents.

0.28

EMooly: Supporting Autistic Children in Collaborative Social-Emotional Learning with Caregiver Participation through Interactive AI-infused and AR Activities

cs.HC ·2026-09-21 ·Yue Lyu, Di Liu, Pengcheng An, Xin Tong, Huan Zhang +2 more
publishedhighly cited similarity 0.61
Abstract · 38 pages. Published in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), Vol. 8, No. 4, Article 2…

Children with autism spectrum disorder (ASD) have social-emotional deficits that lead to difficulties in recognizing emotions as well as understanding and responding to social interactions. This study presents EMooly, a tablet game that actively involves caregivers and leverages augmented reality (AR) and generative AI (GenAI) to enhance social-emotional learning for autistic children. Through a year of collaborative effort with five domain experts, we developed EMooly that engages children through personalized social stories, interactive and fun activities, and enhanced caregiver participation, focusing on emotion understanding and facial expression recognition. Compared with a baseline, a controlled study with 24 autistic children and their caregivers showed EMooly significantly improved children's emotion recognition skills and its novel features were preferred and appreciated. EMooly demonstrates the potential of AI and AR in enhancing social-emotional development for autistic children via prompt personalizing and engagement, and highlights the importance of caregiver involvement for optimal learning outcomes.

0.62

Feel My Focus: Gaze-Triggered Digital Twin Creation for Remote Collaboration

cs.HC cs.CVcs.GR ·2026-11-08 ·Shih-Yu Ma, Suibi Che-Chuan Weng, Ming-Yuan Pai, Tom Yeh, Ellen Yi-Luen Do
top venue similarity 0.60
Abstract · 5 pages, 3 figures

We present Feel My Focus, a gaze-triggered mixed reality (MR) demonstration that enables rapid creation and sharing of lightweight 3D digital twins during remote collaboration. Unlike systems that use gaze primarily as a referential pointer, Feel My Focus treats sustained visual attention as a generative action: a fixation on a physical object initiates single-image capture, attention-guided segmentation, and one-shot 3D reconstruction, allowing a remote partner to inspect and discuss the object in situ. The system integrates gaze-tracking glasses, modular segmentation and reconstruction services, and an MR client for runtime streaming and interaction. This design prioritizes responsiveness and experiential continuity over photometric fidelity. We report observations from a public demonstration, highlighting design trade-offs, failure modes, and user control considerations for deploying gaze-triggered generative systems in real-world collaborative settings.

0.32

Peers4U: Designing Learner-Centered Virtual Peers for Children's Multimodal Mathematics Learning

cs.HC ·2026-10-27 ·Zhi Zheng, Binglin Liu, Weihao Chen, Lan Luo, Yu Mei +3 more
similarity 0.59
Abstract

In early mathematics, children learn abstract concepts through concrete representations, such as manipulating objects, drawing, and speaking. Learning with peers can enrich this process by exposing children to diverse strategies and enabling them to exchange ideas through interaction. We realized this potential with virtual peers in Peers4U, a multimodal learning system to enhance children's mathematical conceptual understanding. Virtual peers driven by LLMs are personalized to the learner's cognitive level and assigned multimodal solutions with inspectable processes. During learning, peers engage in mixed-initiative interactions by responding, initiating topics, and advancing discussions along a pedagogical progression. A user study with 22 children and two experts showed our learner-centered topic orchestration significantly outperformed a persona-centered baseline, and children spontaneously authored new artifacts to demonstrate inspired ideas or share their own strategies, producing significantly more solutions when grouped with more Leading Peers. We contribute design insights for multimodal, learner-centered virtual peer systems.

0.29

Generative Proxy: Synthesizing Proxy-Based Interfaces for Real-World Interaction Across AR Glasses

cs.HC ·2026-08-10 ·Xianhao Carton Liu, William Chastek, Eric J Gonzalez, Mar Gonzalez-Franco, Chen Zhu-Tian
top venue similarity 0.58
Abstract · 11 pages, 11 figures, published in UIST '26, Detroit, MI, USA, Nov 2 - 5, 2026; UIST '26: ACM Symposium on User Interface Software and Techn…

Interacting with real-world objects in AR is difficult, especially when targets are distant, cluttered, or occluded. These challenges are amplified on emerging lightweight AR glasses, which often lack binocular or large field of view on display, but also continuous inputs, such as hand or eye tracking. Proxy-based interfaces offer an alternative by allowing users to interact with virtual abstractions of physical objects that can be repositioned, reorganized, and adapted to the task and device. However, designing such interfaces is currently manual and highly device-specific. We present Generative Proxy, a method for automatically generating proxy-based interfaces from three specifications: scene, intent, and device capabilities. We formulate generation as a constrained synthesis problem that first produces valid interfaces for the target device and task, then ranks candidates using semantic and articulatory distance inspired by direct manipulation theory. We demonstrate Generative Proxy across diverse scenes, device profiles, and user intents. Expert evaluation shows initial evidence that generated proxy UIs are useful and usable, highlighting proxy-based abstraction as a promising interaction paradigm for future AR glasses.

0.40

CoBranchMR: Supporting Parallel Design and Conflict Resolution in Mixed Reality

cs.HC ·2026-09-23 ·Niloofar Sayadi, Kaiyuan Tang, Yunhao Xing, Simret Gebreegziabher, Chaoli Wang +1 more
published similarity 0.58
Abstract · 3 pages, 2 figures, accepted as a Demonstration paper at CSCW 2026

We present CoBranchMR, a mixed reality (MR) system that enables distributed collaborators to work in parallel from different locations on the same digital representation of a physical object. CoBranchMR lets users branch an object into editable virtual copies, customize them independently, and then merge their work back into a shared object. When merging copies, the system displays potential conflicts on the object's surface and provides several resolution options. By adopting branch-and-merge workflows for embodied spatial collaboration, CoBranchMR introduces a new collaborative interaction model that supports parallel design, conflict resolution, and negotiation in remote creative work.

0.34

Generative Tutorial: Towards Live Contextualized Visual Instructions for Physical Tasks

cs.HC cs.AI ·2026-09-21 ·Muzhe Wu, Zuchen Li, Xu Wang, Anhong Guo
similarity 0.57
Abstract

Visual instructions for physical tasks are typically authored in one context and followed in another, requiring users to translate demonstrated tools, materials, and spatial relationships into their own environment. We introduce Generative Tutorial, a conceptual framework for live visual instruction that depicts intended outcomes and actions within the user's environment and task flow. A formative evaluation of state-of-the-art image and video generation identifies failures and potential benefits across 15 physical tasks. Drawing on these findings, we build an augmented-reality prototype system that proactively generates goal images and demonstration videos using observed workspace context and predicted visual outcomes of preceding actions. A 24-participant lab study found higher task performance quality, greater perceived workspace correspondence, and shorter step-confirmation intervals with the system than with pre-authored guidance. Qualitative findings highlighted how contextual resemblance shapes trust, how generation errors affect interpretation, and how guidance delivery should adapt to users' needs, informing future designs.

0.41

Immersive Commons: Designing Ephemeral Spatial Dialogue for Co-Located Mixed Reality Environments

cs.HC cs.MM ·2026-10-26 ·Wuwu Ran, Mohammed Faraj Al-Suwaidi
published similarity 0.57
Abstract · 14 pages, 8 figures. Accepted at SIGraDi 2026, the Conference of the Iberoamerican Society of Digital Graphics

Head-mounted mixed reality (MR) often prioritises individual immersion, potentially drawing attention away from others sharing the room. This paper asks how MR can support co-located spatial dialogue through embodied interaction and activity visible to non-headset observers. We introduce Immersive Commons, a design framework, and Atmos Messages, a prototype combining gesture-based, ephemeral spatial messages with voice transcription, preset text, and a public floor-plan overview. Following a Research through Design approach, fifteen participants created 236 messages across two sessions, including 48 voice-authored messages. Questionnaire responses highlighted playfulness and social awareness, while observations suggested opportunities for exchange grounded in the architectural setting. Interaction clustered near the public display, although this was also where participants put on their headsets. These findings suggest the potential of ephemeral spatial messaging to support co-located exchange, with playfulness, shared visibility, and architectural context informing the design of more social and situated MR experiences.

0.22