CS Paper Radar

Exploring Object Recall in an Object-Rich AR Office Scene

Sangita Kunapuli, Shane Dirksen, Radha Kumaran, Yanxiu Jin, You-Jin Kim, Misha Sra, Tobias Höllerer

cs.HC ·submitted 2026-10-13 11:24 UTC · v1
published

Abstract

Augmented reality (AR) changes how users perceive and attend to their surroundings. Prior work suggests AR can reduce attention to the physical world, but it remains unclear whether this holds in environments dense with both virtual and physical objects, and which factors most affect object recall. This study examines recall in a mixed-reality office scene after an object-selection task, testing object-level factors (virtuality, size), context-level factors (semantic congruency, saliency), and user-level factors (cueing, dual tasking). Cueing informed participants of a later recall test, while dual tasking added cognitive load through a secondary audio task. Virtual objects were recalled more accurately than physical ones. Cueing produced a modest improvement in recall of present objects. Using playback software to reconstruct each participant's view throughout the trials, we computed object-level saliency and found a weak positive correlation with recall, slightly stronger than the effects of virtuality, but not reaching statistical significance. These findings have implications for AR interface design and for building mixed-reality environments that better support cognition during real-world tasks.

Comment
11 pages, 9 figures. Accepted to IEEE ISMAR 2026 and published in IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 32, 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Spatial Memory for AR Glasses: Hierarchical Visual Acquisition and Source-Linked Recall

cs.AI ·2026-11-03 ·Tianxing Zhu, Dan Huang
similarity 0.65
Abstract

We present a spatial-memory system that enables optical see-through AR glasses to recall earlier physical encounters. The dual-camera prototype combines hierarchical visual acquisition with source-linked storage and staged retrieval. Quality screen-ing, global and regional appearance novelty, and temporal support select monochrome observations for detailed RGB cap-ture. Semantic observations retain links to their source images, allowing retrieval to answer indexed historical questions direct-ly or inspect visual details omitted from scene descriptions. The architecture separates capture acceptance, interpretation out-come, and support for a particular question. This distinction motivates two core control mechanisms: bounded reassessment of familiar views when earlier processing leaves an evidence gap, and retention guided by source dependencies. In a 131.35 s egocentric replay, the implemented acquisition policy selected six RGB views from 648 monitoring observations, while feature extraction ran on 97.7% of observations. These results demon-strate selective RGB acquisition and identify frequent local inference as a remaining cost. Exposure duration and cross-view timing further constrain the evidence preserved from se-lected encounters. The system provides a practical acquisition-to-recall architecture for persistent AR assistance, with explicit boundaries between captured evidence, indexed interpretations, and supported historical claims.

0.11

Audiovisual Disparities in VR: Impact on Spatial Perception

cs.GR ·2026-11-06 ·Edurne Bernal-Berdun, Mateo Vallejo, Qi Sun, Ana Serrano, Diego Gutierrez
top venue similarity 0.62
Abstract · 2025 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

Virtual reality (VR) experiences often leverage rich and spatialized multimodal environments to increase immersion and engagement. This demands a consistent spatial perception of audiovisual stimuli, since perceived discrepancies can disrupt the sense of presence. In this work, we investigate the consequences of two types of spatial audiovisual disparities: true disparity, where there is a measurable spatial offset between auditory and visual cues, and perceptual disparity, where users report misalignment despite cues being colocated. Unlike most previous studies that employed controlled but simplified experimental setups, our research focuses on complex, realistic VR environments, allowing us to assess the actual implications for VR content design. Our experiments indicate that users are highly sensitive to true audiovisual disparities in controlled environments, detecting even minor misalignments. However, when engaged in additional tasks within realistic settings, their ability to notice such discrepancies diminishes significantly. We also observed that previously found perceptual disparities persist in complex audiovisual environments. However, we identify self-initiated head rotations as a key factor; its absence prevents the effect entirely. We hope our findings offer practical insights for designing more immersive and perceptually coherent VR experiences.

0.58

Moving Beyond the Flat Screen: Opportunities and Challenges of Extended Reality for Studying Human Brain and Behavior

cs.HC ·2026-10-13 ·Tom Bullock, Radha Kumaran, You-Jin Kim, Emily Machniak, Tobias Höllerer +1 more
published similarity 0.62
Abstract · 4 pages, 1 figure. Workshop paper presented at the NeuroXR 2026 Workshop at IEEE ISMAR 2026 (ISMAR-Adjunct)

Much of what we know about human perception and cognition comes from austere experimental settings: static stimuli presented to sedentary observers on flat monitors, complex responses reduced to simple button presses, and brain activity recorded from immobilized heads using EEG or fMRI. These highly controlled paradigms have provided remarkable insights, but capture a small and often unusual slice of behavior, raising questions about how well the findings generalize to the real world. Extended reality (XR), spanning both immersive virtual reality (VR) and augmented reality (AR) that overlays digital content on the real world, offers a way beyond these constraints, supporting controlled, reproducible experiments in which participants actively engage with three-dimensional tasks. Drawing on a growing body of VR and AR studies of attention, memory, and language -- our own and others', some involving EEG -- we outline what XR makes newly possible for cognitive neuroscience and reflect on the practical and conceptual obstacles that remain. The opportunities are substantial, but realizing them requires solving practical problems of hardware, timing, and measurement, and being realistic about what a virtual or augmented world can and cannot stand in for.

0.40

Better Situational Awareness in AR-HRC? A Comparative Study of Augmented Reality and Mobile Interfaces for Human-Robot Collaboration

cs.HC ·2026-09-01 ·Zhehan Qu, Christian Fronk, Jaewoong Jeong, Pavel Manakhov, Maria Gorlatova
similarity 0.61
Abstract · 11 pages, 9 figures, 1 table. Zhehan Qu and Christian Fronk contributed equally

Augmented reality (AR) facilitates human-robot collaboration (HRC) by enabling in-situ spatial visualizations of the robot and the joint task. However, in safety-critical HRC scenarios such as search-and-rescue, spatial visualizations may also reshape visual attention in ways that create competing situational awareness (SA) demands, potentially introducing new safety concerns. While prior AR-HRC work suggests potential benefits for SA, rigorous evaluations that jointly consider robot and environmental awareness across multiple levels of SA remain limited. We address this through a between-subjects study with 30 participants comparing custom AR and mobile interfaces presenting equivalent information, measuring robot and environmental SA with the Situation Awareness Global Assessment Technique (SAGAT) across all three levels, with concurrent eye tracking to identify the attentional mechanisms underlying any SA differences. Both interfaces achieved high usability; relative to the mobile baseline, AR improved perception-level awareness of the robot but yielded no gains in higher-level robot awareness or in environmental awareness at any level. Gaze analysis explained this: AR freed attention from the map, but that attention was re-invested in the conformal visuals rather than the physical environment. Freeing the eyes from a screen is not the same as directing them to the world, a distinction AR interfaces for safety-critical HRC must design around.

0.30

Harnessing Vision-Language Models for Perceptual Quality Assessment and Autonomous Content Adjustment in Augmented Reality

cs.CV ·2026-09-30 ·Elias Rotondo, Lin Duan, Yanming Xiu, Sangjun Eom, Conrad Li +1 more
publishedbenchmarkcode/data similarity 0.60
Abstract · To be published in VRST 2026. Main Manuscript: 12 pages, 5 figures; Supplemental Materials: 7 pages, 10 figures. The accompanying public rep…

Advancements in augmented reality (AR) continue to foster innovative solutions, facilitating novel methodologies within educational systems, healthcare delivery, and risk-mitigation protocols. However, optimizing for end-user immersion and comfort remains challenging, as AR head-mounted displays contend with constrained scene geometry, spatial jitter, and temporal instability. User studies are the standard AR evaluation method for visual quality, but their cost, diminishing scalability, and inflexibility pose bottlenecks during iterative application design. To address this problem, we present an automated framework for AR content evaluation and refinement, built on vision-language models (VLMs), to evaluate and predict the visual fidelity of AR scenes as perceived by users. First, we introduce RateAR, a benchmark of AR images and videos collected across diverse scenes and environmental conditions, with good-to-excellent reliability (ICC(2,5) >= .90) across perceptual factors, including object placement, scale, and shadow consistency. Subsequently, we evaluate eleven commercial VLMs on the crafted benchmark. Results support that VLM-based quality predictions strongly correlate with human subjective judgments, achieving Spearman's rank-order correlations of up to 0.8695. An ablation study further suggests that, compared to other prompting strategies, our contextual prompting yields better alignment with human ratings while balancing introduced complexity cues. Building on these findings, we construct an automated AR content adjustment system and conduct a 21-participant user study. More than 90% of participants found that the system improved placement and size coherence of virtual content.

0.56

Personalised 3D Mnemonics for Vocabulary Learning in Context

cs.HC ·2026-10-29 ·Karolina Trajkovska, Matjaž Kljun, Maheshya Weerasinghe, Klen Čopič Pucihar
similarity 0.59
Abstract

Augmented reality (AR) can support vocabulary learning by placing mnemonic cues in the physical contexts where new words are encountered. Prior work has augmented real objects with bilingual labels, audio, keywords, and 3D visualisations of those keywords interacting with the objects, but these cues are typically predefined by system designers. This limits scalability and reduces learners' ability to construct personal mnemonic links. By grounding mnemonics in objects that learners already recognise, use, and associate with prior experiences, a system could connect new vocabulary to existing mental models rather than requiring learners to adopt designer-defined associations. We outline a proof of concept and a storyboard of a generative AR approach in which learners create personalised 3D mnemonics by transforming familiar objects in their surroundings into situated mnemonic anchors. We position this concept within a timeline of related work and argue that recent advances in generative 3D asset creation enable a new design point: personalised 3D mnemonics that do not merely place content in the physical learning environment, but modify familiar real-world objects so that they become part of the mnemonic itself.

0.28

Never Look Back: Understanding Persistence in 3D Object Memory from Egocentric Videos

cs.CV cs.AIcs.RO ·2026-10-07 ·Shravan Chaudhari, William Paul, Suchi Saria, Rama Chellappa, Homanga Bharadhwaj
code/data similarity 0.58
Abstract · Project page: https://ledger-3d.github.io . Code: https://github.com/LEDGER-3D/LEDGER

As we move through the world and carry out everyday tasks, we encounter objects that may become relevant only later. We are capable of recalling where we left something or what was inside a container, even without knowing we would need it later. Here, we study how an embodied assistant can build a similar memory from egocentric videos, by observing a person's day-to-day activities. We present Ledger, a persistent 3D object memory that combines object locations, their histories, and contextual descriptions. It associates observations across the recording and retains objects after they leave the view, including those the person never touches. It clusters each object's observations by resting locations and records a move only after repeated evidence, reducing the effect of localization noise. Short descriptions preserve details such as an object's contents or supporting surface. It saves these records to later answer spatial questions without having to access the original images or video. Our memory raises HD-EPIC accuracy from 29.7% to 42.6%, UCS-Bench accuracy from 33.8% to 38.5% and localizes Ego4D objects with a 0.99 m median error on returned predictions. Our analyses identify complementary roles for temporal persistence, contextual descriptions, and retrieval. Our study on 100 stitched streams of multiple scenes each further exposes failures in both retrieval and construction. Per-scene construction partially recovers the performance lost across scene changes compared to that of single scene streams.

0.47

FocusAdapt: Context-aware Adaptive Focus Assistance in Diminished Reality

cs.HC ·2026-08-31 ·Tianyu Zhang, Shutong Wu, Jiankun Yang, Zhen Bai, Yukang Yan
published similarity 0.58
Abstract

Diminished Reality (DR) can reduce visual clutter by removing irrelevant objects. However, removing all task-irrelevant objects may eliminate useful contextual information and reduce situational awareness. We present FocusAdapt, a context-aware DR system that predicts object-level distraction by integrating visual saliency, semantic relevance, and gaze behavior. Based on findings from a formative study, FocusAdapt selectively diminishes highly distracting objects while preserving useful context, enabling adaptive focus assistance during procedural tasks.

0.42