CS Paper Radar

Think Once, Reflect Twice: How Feedback Shapes Human-AI Collaboration

Joshua Holstein, Philipp Spitzer, Sebastian Schäfer, Maximilian Förster, Gerhard Satzger

cs.HC ·submitted 2026-10-14 11:09 UTC · v1
hot direction

Abstract

As artificial intelligence (AI) increasingly supports human decision-making in high-stakes domains, designing effective human-AI collaboration becomes critical. Successful collaboration requires humans to reflect on decision-making processes and adjust their strategies accordingly. Current research typically investigates interventions, such as explainable AI, that target reflection during the decision-making process (reflection-in-action). Less attention has been given to retrospective examination of decisions (reflection-on-action). This work explores reflection-on-action by studying the impact of feedback on reliance behavior and human-AI team performance. Specifically, we investigate feedback interventions targeting overrides of AI advice through a behavioral study (N=244). We find that such feedback influences reflection-on-action, while unidirectional feedback risks reducing subsequent reflection-in-action. Furthermore, we identify opposing shifts in reliance behavior that leave overall human-AI team performance unchanged but show context-dependent effects. Balanced feedback did not show these risks, preserving both concurrent and retrospective reflection, offering insights for designing for appropriate reliance on AI.

Closest papers

Nearest neighbours by meaning across the whole index.

Novice Reliance Calibration in AI-Assisted Decision Making: The Role of Explanations and Self-Assessment

cs.HC cs.AI ·2026-10-06 ·Eun Jeong Kang, Peter, Duan, Swati Mishra
similarity 0.65
Abstract · under review

Artificial Intelligence (AI) tools are widely used to support decision making in tasks and domains where no immediate performance feedback is available. In these settings, users cannot learn to adjust their reliance behavior over time through trial and error. However, little is known about how novice users calibrate reliance on AI when external feedback is unavailable, or whether AI explanations can support calibration in its absence. We introduce reliance calibration as an organizing construct for studying how novice users dynamically adjust reliance behavior, and examine how AI explanations and meta-cognitive self-assessment shape it. Through a between-subjects study with 110 participants completing a clinical entity extraction task with AI assistance and limited performance feedback, we observe that novice users exhibit systematic drift toward over-reliance in the presence of explanations, while higher self-reported task understanding is associated with more selective reliance behavior. These results extend reliance calibration research into human-AI collaboration contexts without real-time performance signals and present actionable guidelines on designing AI tools that must support appropriate reliance in these settings.

0.29

Why This and Not That? A Collaborative Reflection Approach for Understanding Thought Coverage in Decision Making Support Dialog

cs.HC ·2026-08-17 ·Morita Tarvirdians, Hayley Hung, Catharine Oertel
top venue similarity 0.62
Abstract · Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026

Conversational agents that support reflection for decision-making often rely on adaptive dialog policies that map observed user behavior to actions such as probing, deepening, or redirecting. Yet the same pattern can reflect a range of different reasons such as deliberate prioritisation or limited self-access. By modeling the observable pattern rather than the user's reason for it, current policies risk premature assumptions about the user state and inappropriate next actions. To address this gap, we introduce a human-centered method for surfacing this hidden inference step. In a user study with 62 users and 232 collaborative moments, we pause a reflection-support agent when it would normally redirect the conversation, surface its observation, and ask users to interpret the pattern and decide how to proceed. We derive a taxonomy of nine interpretation categories and show that similar reflective states can call for substantially different follow-up actions. Our findings challenge the assumption that adaptive dialog policies can rely on observable behavior alone, and suggest how user-provided interpretations can inform more appropriate conversational actions.

0.46

Following Delegation Traces to Study Human Oversight in Interaction with Multimodal AI

cs.HC cs.AIcs.CV ·2026-10-15 ·Agnese Chiatti, Matteo Matteucci, Viola Schiaffonati
survey similarity 0.61
Abstract

As multimodal AI systems are becoming our everyday thought partners, the shift towards studying how users transfer control to these systems becomes especially urgent. In this paper, we report an overview of findings from an online study with n=56 AI-literate participants and 392 Video Question Answering trials. After inspecting each multimodal AI response, participants rated task-specific trust and selected one of three intended policies for an analogous future task: reject AI assistance, use it with verification, or delegate fully to the AI. Within participants, post-interaction trust strongly predicted the user's delegation choices (OR=14.97, 95% CI [7.14,31.37]). Trust and delegation nevertheless remained distinct. Verification was the dominant policy and often co-occurred with high trust. Delegation also increased with task order after adjustment for momentary and participant-level trust (OR=1.22, 95% CI [1.02,1.45]), indicating an experience-related component that self-reported trust ratings alone could not capture. These findings motivate a shift from intended delegation to a non-intrusive observation of how delegation is enacted in collaborative human-AI tasks. We therefore conclude by proposing enacted delegation traces as a methodological framework for future studies of multimodal and agentic AI use.

0.13

Improving Group Deliberation through AI-Guided Self-Reflection

cs.HC cs.CY ·2026-10-27 ·Haiwen Li, Crystal Qian, Ben Usman, Stephanie Chan, Michiel Bakker +7 more
similarity 0.61
Abstract

As AI systems increasingly mediate human communication, identifying conditions under which they augment collective sense-making is a critical challenge. Through a randomized experiment (N=1,173 U.S. participants across 391 small-group deliberations), we investigate whether AI-guided self-reflection improves subsequent deliberative discourse. We show that engaging with a non-directive AI chatbot instructed to structure self-reflection prior to group deliberation yields significantly higher deliberation quality (measured by the Discourse Quality Index, DQI) compared to a baseline of no self-reflection (Cohen's d=0.31) and a condition of unassisted self-reflection (d=0.27), and the positive impact is most pronounced in groups with disagreement. This improvement may be driven by more sophisticated argument justifications, greater consensus-building effort, and increased respect for counterarguments. Furthermore, after self-reflection, participants in the AI condition reported higher empathy toward dissenting peers and a deeper understanding of opposing reasoning. Beyond observed behavior, our findings are empirically relevant to open debates between political philosophy and psychology surrounding self-reflection, demonstrating that interactive AI could transform solitary thinking into productive internal reflection. Together, these results demonstrate that even simple AI-based reflective scaffolds can foster more productive group dynamics, opening new avenues for leveraging AI to augment collective intelligence.

0.42

Thinking Twice: A Sequential Approach to Nudge Towards Reflective Judgment in GenAI-Assisted Decision Making

cs.HC ·2026-11-06 ·Hüseyin Hussein Keke, Daniel Eisenhardt, Christian Meske
published similarity 0.60
Abstract · Full paper accepted and presented at the 20th International Conference on Wirtschaftsinformatik (WI 2025), Münster, Germany. To appear in th…

Humans increasingly use generative AI (GenAI) for decision-making assistance, yet tend to engage in such situations using quick, intuitive (System 1) rather than deliberate, analytical assessment (System 2) as reflected in Dual Process Theory. Such engagement reduces the effectiveness of human-GenAI interactions and increases associated risks. Our research investigates how to design human-GenAI interactions that promote the transition from System 1 to System 2 thinking in decision-making. Based on the Three-Stage-Model, which suggests humans switch to System 2 when conflicts are detected, we conducted an experiment (n=130) testing sequential human-GenAI interaction as a cognitive forcing nudge during problem-solving tasks. This approach, where humans decide first before receiving AI assistance, enables detection of conflicting decisions. Compared to non-sequential settings (concurrent or no AI assistance), results show the sequential condition significantly enhanced both decision performance and GenAI prompt utilization, providing valuable insights for designing more reflective and thus effective human-GenAI interactions.

0.24

Goals as a Representational Layer for Human-AI Collaboration

cs.HC cs.AI ·2026-11-05 ·Gaia Molinaro, Lev Tankelevitch, Sean Rintel, Scott Counts
position similarity 0.60
Abstract

Goals are central to how people, teams, and organizations structure their work. Yet, while generative artificial intelligence (AI) systems increasingly operate across a worker's context, information about why a person is pursuing a task, beyond local prompts and interactions, is typically missing. We argue that goals should become an explicit representational layer mediating interactions among people, AI systems, and organizations. Because human goals are hierarchical, partially latent, and dynamically evolving, such a layer must be multidimensional, both grounded in human articulation and inferred from available evidence, and continuously updated. A goal layer could connect immediate requests to broader outcomes, improve the organization and retrieval of relevant context, support reflection and coordination, and provide a foundation for training, aligning, and evaluating AI systems in terms of human-centered outcomes. We highlight these opportunities while identifying open challenges involving goal inference, socio-cognitive frictions, alignment, privacy, and governance.

0.23

Open Questions Towards Skill-Sustaining Reliance in Reflective AI Engagement

cs.HC ·2026-09-23 ·Sander de Jong
similarity 0.60
Abstract

As AI systems are increasingly integrated into professional work, reflection strategies such as cognitive forcing and prompts that foster critical engagement have shown promise in reducing overreliance and improving decision quality. However, these strategies have primarily been evaluated as short-term interventions within single sessions. The next challenge is to assess whether such mechanisms sustain human agency and expertise over time. Drawing on prior work in AI-assisted decision-making, metacognition, and reflective AI engagement, we examine the challenges of designing and evaluating reflective mechanisms for long-term skill sustainability, considering individual differences in how users engage with such support, the organisational conditions under which it is implemented, and the gap between short-term evidence and long-term claims. We introduce open questions for the research community about the conditions under which reflective AI engagement can be sustained in practice.

0.14

Same Team Label, Different Evidence: A Full-Text Audit of Claim Denominators in Human-AI Teaming Research

cs.HC ·2026-08-25 ·Hanjing Shi, Kimberly Wang, Sabrina Doherty, Dominic DiFranzo
similarity 0.59
Abstract · 22 pages, 4 figures, 6 tables. Preprint

Human-AI Teaming (HAT) reviews often group studies by labels such as advisor, teammate, or coordinator. Yet the same label can describe one person taking AI advice, several people coordinating around AI, or a workflow that distributes authority and responsibility. Pooling these studies can therefore change the human unit behind a claim. We examine how full-text evidence changes the set of studies behind a claim. We audited 86 full texts purposively selected from a 419-record title/abstract map. We find that full-text reading changed core membership for 40 records: 36 of 74 apparent core candidates moved out, while 4 of 12 boundary candidates moved in. Team vocabulary did not reliably identify the social unit: 14 of 27 human-AI dyads and 20 of 23 multi-human peer teams used team or collaboration terms. Only 20 of 86 papers specified who could see AI output. Four blinded language-model runs unanimously labeled 53 screening cases and 59 arrangements, yet 32% and 34% of those consensus decisions differed from the full-text labels. These results identify claim-denominator drift as a synthesis problem in HAT research. We contribute a full-text audit centered on human arrangements and a claim-pooling checkpoint for deciding when evidence about trust, coordination, performance, efficiency, and accountability can be compared.

0.21