CS Paper Radar

Seeing through the Eyes of AI: Situated Explainability in Augmented Reality

Ana Stanescu, Lucchas Ribeiro Skreinig, Tobias Langlotz, Stefanie Zollmann, Peter Mohr, Dieter Schmalstieg, Mark Billinghurst, Denis Kalkofen

cs.HC ·submitted 2026-10-02 12:43 UTC · v1

Abstract

Explainable Artificial Intelligence (AI) enables humans to understand and interpret decisions of AI models. Instead of having a black box, explainability supports humans in understanding AI models' behavior. Existing explainable AI approaches often present explanations on 2D displays using pre-recorded data, requiring users to relate the displayed information back to the physical objects and real world locations involved in a model's decision. Users are forced to decouple data exploration and capture from AI model interpretation. For AI systems that work within physical environments, this separation can make explanations difficult to interpret in context. We propose using Augmented Reality (AR) to enhance the understanding of AI models by enabling spatial explainability information directly in a user's workspace, in real time, as they explore the world. We show how known explainability methods can be applied in AR and provide insights into user experiences with such an application.

Comment
11 pages, 8 figures

Closest papers

Nearest neighbours by meaning across the whole index.

How Contextual Information in Physical Environments Shapes Decisions with AI

cs.HC ·2026-11-02 ·Omar Elgohary, Amanda Elvarsdottir, Vannessa Lee, Zhu-Tian Chen
similarity 0.64
Abstract

AI assistants in augmented reality may advise users about parts of the physical world they cannot directly observe. We ask whether contextual information from the situation helps users make better decisions with fallible AI. In an AR visual- search study (N = 32), we varied target prevalence and scene visibility while participants judged whether a key was present, with or without AI assistance. Lowering target prevalence and revealing part of the desk together improved observed AI-assisted accuracy, but our analysis did not demonstrate that this gain came from participants making better use of context. Context also changed which answers seemed plausible: when keys were rare, participants became more skeptical of Present suggestions, but this shift applied to both correct and incorrect suggestions rather than improving selective acceptance of correct advice. Revealing part of the desk also increased decision time, with little evidence of better AI use. Overall, contextual information can shape AI- assisted decisions and increase decision effort without necessarily helping users judge when the AI is correct.

0.13

Adapting Vision-Language Models for Human-Readable XAI in Industrial Object Detection

cs.CV cs.AI ·2026-09-17 ·Sarvenaz Sardari, Freddy Fernandes, Samarth Yelvande, Jose Moises Araya-Martinez, Alina Roitberg
benchmark similarity 0.59
Abstract · Procedia CIRP ICME '2025

Explainable Artificial Intelligence (XAI) solutions are essential for building trust in AI technologies and their integration in real manufacturing lines. However, most existing methods are tailored to technical experts, limiting their accessibility to diverse user groups such as blue-collar workers in manufacturing lines who use AI for quality control. In this work, we introduce an XAI interface for object detection in industrial manufacturing based on a fine-tuned vision-language model, designed to generate intuitive explanations for non-expert users. We benchmark existing vision-language models and demonstrate that out-of-the-box models often fall short in delivering clear, context-relevant explanations for non-expert users. To address this, we fine-tune a vision-language model and integrate it into our interface, enabling contextualized, accessible explanations for non-expert users. We demonstrate improvements in explanation clarity, instruction adherence, image groundedness, and contextual awareness over GPT 4o-mini on proprietary and public robotics dataset. This approach advances the accessibility and usability of AI explanations, making them more intuitive and applicable in manufacturing domain.

0.23

Context Matters! Social, Spatial, and Semantic Coordination with an Embodied Agent in Immersive Analytics

cs.HC ·2026-10-15 ·Andrew Cunningham, Nicolas Barbotin, Adam Drogemuller, Jack Faser
top venue similarity 0.59
Abstract · In 2025 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) (pp. 189-194). IEEE

Embodied AI agents in Augmented Reality (AR) could enable more natural collaboration in immersive analytics by engaging users through spatial, conversational, and visual interaction. However, designing such agents requires careful attention to context. We propose a preliminary conceptual framework for context-aware design of embodied agents structured around three dimensions of spatial, social, and semantic. This framework is further categorised into embeddedness (how agents are situated in the contexts) and awareness (how agents perceive and respond to the contexts). To explore this, we present Cobot, an embodied AI agent designed as a design probe. Cobot supports context-sensitive interaction through features such as gaze cues, spatial sound, and task-aware behaviour. We reflect on its design and identify implications for future contextaware embodied agents.

0.28

A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods

cs.AI ·2026-09-24 ·Miquel Miró-Nicolau, Francesco Spinnato, Riccardo Guidotti
similarity 0.59
Abstract

Evaluating explainable Artificial Intelligence (XAI) methods is a challenging task due to the lack of reliable evaluation procedures and, in particular, the absence of ground truth explanations. In the literature, existing evaluation approaches typically assess explanations by measuring their fidelity with respect to the predictions of a black-box model. However, such evaluation strategies only quantify the degree to which an explanation reproduces the model's output, without ensuring that the explanation correctly reflects the underlying decision process. As a consequence, different explanations may achieve similar fidelity scores while providing inconsistent or misleading interpretations of the model behavior. In this paper, we propose a framework for the evaluation of XAI methods based on synthetic ground truth. The proposed approach relies on controlled interventions to generate synthetic datasets in which the importance of input components can be determined by design. This enables the construction of ground truth explanations that are directly aligned with the behavior of the model under analysis. The framework is instantiated across three data domains, namely binary images, tabular data, and time series, allowing a comprehensive assessment of explanation methods in heterogeneous settings. Experimental results obtained by evaluating nine widely used XAI methods show significant limitations in current techniques and highlight the importance of synthetic, intervention-based benchmarks for a reliable assessment of explanation quality.

0.14

PiPS: Post-Hoc Prototypical Explanations for Interpretable Semantic Segmentation

cs.CV ·2026-09-15 ·Miłosz Adamczyk, Tymoteusz Zapala, Piotr Borycki, Przemysław Spurek
code/data similarity 0.58
Abstract

With the increasing deployment of deep neural networks in critical systems, such as medical diagnostics and autonomous vehicles, ensuring their interpretability is crucial to building trust in decision-making systems. In the field of explainable artificial intelligence, prototype-based reasoning has gained particular popularity, as it mimics human cognitive processes by explaining model decisions based on visual similarity under the looks like this paradigm. While this paradigm has been thoroughly investigated in the context of global image classification, the interpretability of dense predictions, particularly semantic segmentation, remains largely unexplored despite its immense importance in tasks requiring precise object localization. Existing prototype-based interpretable segmentation models rely on ante-hoc architectures, which entails significant limitations because they require costly training from scratch and modifications to the network structure, ultimately leading to a noticeable drop in predictive performance compared to standard black-box models. To address this issue, we propose PiPS (Post-hoc interpretable Prototypical Segmentation), the first fully post-hoc solution for generating prototypical explanations for semantic segmentation models. Our method enables the extraction of intuitive, spatially localized explanations from any pre-trained network without modification or fine-tuning, thereby preserving 100% of the model's original predictive performance. This approach opens a new avenue for the safe and cost-effective deployment of transparent systems in advanced computer vision tasks. Codebase available at https://github.com/gmum/PIPS.

0.35

RealityBench: Simulated Environments to Evaluate a Vision Language Model's Ability to Document Human Physical Work

cs.CV cs.AIcs.CL ·2026-10-09 ·Andrew Perrault, Anmol Takiar, Dominik Winecki, Arnab Nandi
benchmarkcode/data similarity 0.58
Abstract · 28 pages, 3 figures, 14 tables. Data: https://huggingface.co/datasets/relantic/realitybench

Augmenting worker capabilities in the physical world through guidance has been a decades-long goal for AI agents. One key supporting capability is to understand what the worker is doing, seeing through the human's camera or smart glasses, and assist them in documentation. Despite advances in vision language models, their capabilities in these settings remain uncertain and are challenging to benchmark because most physical work data is proprietary, and involves too much variability to study in a principled manner. Unlike structured settings such as kitchens and warehouses that have predictable lighting, geometry, and objects, field-facing tasks such as rooftop solar installation and vehicle repair are unconstrained and high variance. Here, we model real-world physical work in Blender across eleven environments, incorporating variants in both image acceptability and documentation. The acceptability task asks whether a particular view would be suitable proof-of-work and documentation asks whether an agent can understand the task-relevant state of the environment from that view. Across 26 vision-language models, we find that top models largely solve documentation when the answer is text in frame, but are weak at reading measurements and at judging small parts and spatial relations, such as whether a tape measure leans. Acceptability is harder to pin down because human raters disagree with each other. Measured against a scorer that agrees with the raters' majority about as often as a single rater does, the best models cannot be distinguished from a single non-author rater, at a cost within the range of a human pass. In the three environments where we can trace acceptability errors, most wrong rejections come from misreading the scene rather than the requirement. On the three documentation questions for which we have real images, model rankings on rendered frames transfer to real images.

0.44

SituRecBench: Evaluating Frontier Models as Situated Recommendation Assistants in 3D Environments

cs.AI cs.CLcs.CVcs.IR+1 ·2026-11-07 ·Jingtong Yang, Dongding Lin, Jian Wang, Wenjie Li
top venuebenchmark similarity 0.58
Abstract · Accepted by NeurIPS 2026 Evaluations and Datasets Track

Recommender systems are transitioning from static retrieval engines into interactive assistants grounded in physical environments. However, current studies largely neglect the combined complexities of spatial navigation, visual grounding, and multi-turn social interaction. To bridge this gap, we introduce SituRecBench, an interactive benchmark for evaluating frontier vision-language models (VLMs) as situated recommendation assistants within high-fidelity 3D environments. We reformulate recommendation as a situated assistance process in which an assistant interacts with a user while sharing the user's egocentric view. Our benchmark simulates real-world shopping environments (e.g., supermarkets and home furnishings stores), requiring assistants to elicit preferences, ground recommendations in the environment, and guide the user to relevant locations through dialogue. The benchmark evaluates dialogue quality, recommendation utility, and navigation effectiveness through a unified interactive protocol. Extensive experiments reveal persistent challenges in spatial grounding and preference elicitation, highlighting limitations in their ability to integrate global contextual information with local observations. SituRecBench provides a rigorous foundation for advancing the development of situated assistants in the real world.

0.34

Neural Representations as New Modes of Interactivity

cs.HC ·2026-11-12 ·Stanley Huang, Anthony Baez, Catherine Yeh, Pat Pataranutaporn
position similarity 0.57
Abstract · 12 pages, 2 figures, 2 tables

Human-AI interaction today runs almost entirely through prompts and outputs, keeping AI systems opaque to general users. We introduce Neural Representation Interaction (NRI), a paradigm that treats the internal representations of neural networks as an interaction material people can inspect, steer, and edit. Situating NRI alongside graphical, tangible, and bio-computer interaction, we argue that the concepts and algorithms models encode internally offer representational affordances for richer human-AI interaction. We synthesize actionable mechanistic interpretability methods and, drawing on General Morphological Analysis, construct a design space spanning Interaction, Target, Referent, and Purpose. Mapping existing interfaces onto this space reveals that most serve expert auditors, while few support lay users. We then present speculative applications in education, personal user modeling, and hiring audits, and discuss design principles, limitations, and sociotechnical risks. NRI offers HCI researchers abstractions for designing interactions that work with AI computation rather than only through its inputs and outputs.

0.37