CS Paper Radar

Leveraging Mental Models for Human-Centered Explanations: A Systematic Design Approach Illustrated by Explainable Fake Review Identification

Sidra Naveed, Md Shajalal, Gunnar Stevens, Alexander Boden

cs.HC cs.AI ·submitted 2026-10-17 22:56 UTC · v1

Abstract

Explainable AI (XAI) often prioritizes model fidelity over user intelligibility, yielding explanations that satisfy technical transparency yet fail to support human sense-making. We propose a human-centered approach, MIND-X (Mental-model INtegrated Design for eXplainability), which aligns explanations with users' mental models, commonsense concepts, and judgment heuristics. Using fake review detection as a case study, we contribute: (1) a qualitative study (n=10) that elicits how people assess review credibility and produces an actionable, design-guiding taxonomy of perceptual cues; (2) a set of design principles that link those cues to explanation strategies and prompt LLMs to generate adaptive, language-based explanations; and (3) a quantitative evaluation (n=166) showing that these explanations offer distinct advantages over token-level transparency methods widely used in explainable text classification in terms of understandability, satisfaction, perceived trust, persuasiveness, and decision-making support. Rather than claiming general superiority, our evaluation illustrates how grounding explanations in users' everyday reasoning practices can produce more intelligible and context-sensitive accounts. Our findings advance human-centered explanation design beyond a mere "gloss of transparency," demonstrating that a mental-model-aligned design process can improve intelligibility and support accountability across user-centered explanation tasks.

Comment
38 pages, 10 figures, 8 tables. Keywords: Explainable Artificial Intelligence (XAI); Human-centered AI; Mental models; Fake review detection; User-centered explanation design; Natural language explanations; Cognitive heuristics

Closest papers

Nearest neighbours by meaning across the whole index.

XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?

cs.AI ·2026-09-08 ·Yanfei Hu Fleischhauer, Alona Zharova, Nadja Klein, Stefan Feuerriegel
benchmark similarity 0.71
Abstract

Evaluating the quality of explanations produced by explainable AI (XAI) methods remains challenging because existing approaches often rely on subjective human judgment, limiting reproducibility, scalability, and comparability between studies. We examine whether LLMs can serve as a reproducible and scalable mechanism to make comparative assessments of the quality of XAI explanations. We introduce XAI-Arena, an LLM-as-a-judge framework for scalable, reproducible, multidimensional, and stakeholder-sensitive evaluation of XAI explanation quality. XAI-Arena then allows us to compare XAI explanations along various dimensions, namely, perceived simplicity, clarity, task adequacy, trust calibration, actionability, transparency, faithfulness, and overall interpretability. We then benchmark XAI explanation methods across various datasets, machine learning models, and stakeholder personas. Human validation shows a strong positive association between LLM-generated and human ratings (Spearman's rho=.693, p<.001). Together, LLM-based evaluations can capture systematic differences in XAI explanation quality and provide a scalable and reproducible framework for comparative assessment of XAI explanations.

0.16

Explanation Navigator: Rectifying Out-of-Scope Human Interpretations of Leaky AI Explanations through Conversational Guidance

cs.HC ·2026-09-19 ·Yueqing Xuan, Kacper Sokol, Danula Hettiachchi
similarity 0.67
Abstract

As explanations of artificial intelligence systems proliferate, their recipients must grasp not only what they convey but also recognise what they cannot. We conducted an interview study with nine participants to examine how explainees reason when their information needs exceed the scope of available explanations. Participants often unwittingly confabulated explanatory insights when relevant information was missing from the explanations, not recognising the inherent limitations thereof. We characterise such explanations as leaky explanations -- simplifications that strive to hide complexity yet whose correct interpretation hinges on understanding of the concealed details. To address out-of-scope interpretations we propose Explanation Navigator: a conversational interaction framework that detects mismatches between users' information needs and explanations' content, elucidating pertinent yet implicit details and providing complementary explanations for unmet information needs. An online study with 316 participants showed that our approach allowed explainees to recognise and rectify confabulated explanatory insights, guiding them towards developing correct understanding.

0.05

Explaining AI Agents Through Execution Traces

cs.AI ·2026-09-05 ·Vittoria Vineis, Fabiano Veglianti, Lorenzo Antonelli, Claudia Di Carlo, Matteo Silvestri +1 more
similarity 0.66
Abstract

AI Agents are increasingly deployed in real-world settings, where they interact with external tools and make sequential decisions with limited human oversight. This creates a pressing need for reliable and auditable explanations of what an agent did and why. However, traditional Explainable AI (XAI) methods fall short of providing the process-level transparency required for such interactive, multi-step systems, motivating a paradigm shift toward approaches specifically designed for AI Agents. To address this gap, we present a post-hoc XAI framework that transforms a lengthy agent's execution trace into a structured report and a faithful natural-language explanation explicitly grounded in its observable behavior. Because it relies solely on execution traces, the framework applies across different agent architectures, environments, and tasks. Human and automated evaluations across multiple benchmarks and architectures show that our framework produces high-quality, trace-faithful explanations while reliably identifying unsupported claims, unjustified actions, and evidence gaps, outperforming naive LLM-generated explanations.

0.16

A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods

cs.AI ·2026-09-24 ·Miquel Miró-Nicolau, Francesco Spinnato, Riccardo Guidotti
similarity 0.65
Abstract

Evaluating explainable Artificial Intelligence (XAI) methods is a challenging task due to the lack of reliable evaluation procedures and, in particular, the absence of ground truth explanations. In the literature, existing evaluation approaches typically assess explanations by measuring their fidelity with respect to the predictions of a black-box model. However, such evaluation strategies only quantify the degree to which an explanation reproduces the model's output, without ensuring that the explanation correctly reflects the underlying decision process. As a consequence, different explanations may achieve similar fidelity scores while providing inconsistent or misleading interpretations of the model behavior. In this paper, we propose a framework for the evaluation of XAI methods based on synthetic ground truth. The proposed approach relies on controlled interventions to generate synthetic datasets in which the importance of input components can be determined by design. This enables the construction of ground truth explanations that are directly aligned with the behavior of the model under analysis. The framework is instantiated across three data domains, namely binary images, tabular data, and time series, allowing a comprehensive assessment of explanation methods in heterogeneous settings. Experimental results obtained by evaluating nine widely used XAI methods show significant limitations in current techniques and highlight the importance of synthetic, intervention-based benchmarks for a reliable assessment of explanation quality.

0.14

ExCritique: An Interactive Visual Interface to Evaluate AI-Generated Explanations at Scale

cs.HC ·2026-11-09 ·Meheraj Hossain, Avinash Reddy Mudireddy, Naimul Hoque
similarity 0.63
Abstract

Vision-language Models (VLMs) can generate natural language explanations for complex decision-making scenarios. However, these explanations can sometimes be misleading, sounding convincing while containing incorrect or unsupported information. Automated explanation-quality metrics help researchers and practitioners evaluate large collections of explanations, but their scores alone provide limited support for identifying failure cases and understanding the reliability of these metrics. This paper presents a case study on how interactive visualizations can help researchers and practitioners evaluate both AI-generated explanations and automated explanation-quality metrics across large datasets. Based on formative interviews with nine AI researchers and practitioners, we designed an interactive visualization tool named ExCritique, situated in the task of Visual Question Answering (VQA). ExCritique visualizes the distributions of multiple automated quality metrics, allows users to explore thousands of data items, filter items and inspect the evidence behind individual scores. We compared ExCritique with a baseline interface without interactive visualizations in a controlled study involving 16 AI researchers and practitioners. Our findings suggest that the interactive visualizations in ExCritique helped users identify explanation-quality issues more efficiently and better understand automated quality metrics, supporting an effective way to measure the reliability of AI-generated explanations.

0.21

Selecting The Most Informative Tokens in Natural Language Autoencoders

cs.CL cs.AI ·2026-09-29 ·Federico Torrielli, Gianluca Barmina, Andrea Blasi Núñez, Amon Rapp, Luigi Di Caro +2 more
similarity 0.63
Abstract

Natural language autoencoders translate a language model's internal activations into readable explanations. Explaining every token position is costly. Which positions should an auditor inspect to understand a potential threat? We study this question across $4.7$ million explanations on prompt injection and concealment. We compare signals from model computation with a ranker trained only on chat structure. Chat structure usually selects more relevant explanations than the computational signals, without requiring a model forward pass for position selection. On three of four datasets, explaining just $5\%$ of positions retains nearly all of the success rate from explaining every position, where success means obtaining an explanation about the threat. The benefit varies with the audit task. We also show that pretrained verbalizers recover words that models have learned to conceal through fine-tuning, without additional verbalizer training. These results identify where auditors can concentrate explanation generation and show that useful explanations can extend beyond the model a verbalizer was trained to describe.

0.16

MEA: A Reward-Driven Multi-Agent System for Faithful Model Explanations

cs.AI ·2026-10-01 ·Yuyang Cheng, Raghav Kaushik Ravi, Srivarshinee Sridhar, Sriparna Saha, Akash Ghosh +1 more
similarity 0.63
Abstract

Recent years have seen the employment of a plethora of machine learning (ML) models in high-stakes domains, but they remain largely opaque to the practitioners who act on their predictions. While post-hoc explanation methods offer a lens into this model behavior, wielding them effectively demands expertise most domain experts lack: navigating high-dimensional outputs, selecting the best explanations, and synthesizing evidence across disparate tools. To this end, we present MEA, a multi-agent framework that removes the explanation knowledge barrier entirely: a Proposer agent selects and configures explanation tools based on the question and modality, while an Actor agent is optimized end-to-end against faithfulness, transforming the outputs into natural language explanations grounded in model behavior across tabular, text, and vision modalities. Further, we introduce diverse question types spanning feature attribution, counterfactual reasoning, and spurious feature detection, each paired with a perturbation-based faithfulness metric. We find that frontier LLMs systematically produce unfaithful explanations. By optimizing against faithfulness rewards augmented with a modality-adaptive penalty, MEA consistently outperforms post hoc explainers, agentic, and closed-source baselines across six datasets, with reward-driven optimization yielding faithfulness gains of +28% (tabular), +21% (text), and +34% (vision) over the untrained backbone. More broadly, our findings suggest that AI agents themselves can serve as a scalable, adaptable interface to ML explainability, opening a path toward natural-language explainability that generalizes beyond the fixed, single-purpose tools that have long defined the field.

0.17

RAG-Enhanced LLMs for Interactive Explainability in Clinical Decision Support Systems

cs.AI cs.HCcs.LG ·2026-11-06 ·Lorenzo Mannocci, Francesca Naretto, Lucia Passaro, Anna Monreale
published similarity 0.63
Abstract · Published in HC_AIxIA_HYDRA 2025, Communications in Computer and Information Science, vol. 2830, pp. 50-63, Springer, 2026. Please cite the …

Artificial Intelligence (AI) is increasingly embedded in high-risks decision-making systems across domains such as healthcare, finance, and the judicial system. As reliance on predictive models grows, so does the demand for Explainable AI (XAI) to ensure transparency, trust, and interpretability. However, existing XAI approaches often fail to present explanations in a format accessible to domain experts, limiting their practical utility. Large Language Models (LLMs) have recently emerged as a bridge between complex models and end-users, offering natural language explanations. Yet, their use introduces new risks, including hallucinated outputs and lack of source traceability. To address these challenges, we propose a Clinical Decision Support System (CDSS) that not only combines standard predictive models with XAI techniques but also incorporates a Retrieval-Augmented Generation module as an interactive enhancer. This module grounds explanations in verified medical knowledge, reformulates technical outputs into a human-like format, and enables users-such as clinicians-to actively query the system. By fostering an interactive, human-in-the-loop environment, our approach empowers domain experts to explore model decisions, contextualize explanations, and build trust in AI-assisted diagnostics. The study concludes with preliminary experiments validating the proposed methodology.

0.20