CS Paper Radar

AI-Assisted Classification under Correlation Neglect and Trust

Dingwen Kong, Liang Lyu, Saurabh Amin, Amine Bennouna, Daniel Huttenlocher, Asuman Ozdaglar

cs.GT cs.AIcs.LG ·submitted 2026-11-04 03:04 UTC · v1
position

Abstract

When does an AI assistant improve human decision-making, and when does it hurt? We argue that the answer depends not just on AI accuracy but also on how much its evidence overlaps with the human's and how the human combines the two. A worker makes a binary decision by observing her own signal and receiving a recommendation from an AI. We develop a human behavioral model with two primitives: her awareness of the overlap and her trust in the AI. Our analysis gives a closed-form phase diagram over AI capability and overlap, with regions of complementarity, impairment, and automation. A worker who neglects the overlap (correlation neglect) can be made less accurate by a more capable AI, double-counting the shared evidence. Trust reshapes the regions - undertrust may compensate for correlation neglect, so that the trust level maximizing accuracy depends on the deployment parameters. Empirically, we identify the two model parameters by pairing the same workers with different AIs, and break the accuracy loss into a correlation-neglect cost and an undertrust cost, one tied to reducing overlap on the AI side, the other to calibrating trust on the human side. On ChatBench, AI assistance raises accuracy by 21 points, yet workers apply only about a quarter of the Bayes-rational weight to AI information; GPT-4o falls in the automation region and Llama-3.1-8b stays at the complementarity boundary; and calibrating trust recovers about 2.6 points for GPT-4o but almost nothing for Llama-3.1-8b.

Closest papers

Nearest neighbours by meaning across the whole index.

Trust Hinders Bidirectional Human-AI Alignment

cs.AI cs.HC ·2026-10-10 ·Michaela Benk, Kacper Sokol
publishedpositionhot direction similarity 0.67
Abstract · 40th Conference on Neural Information Processing Systems (NeurIPS 2026). Workshop: Dynamic Alignment in Human-AI Coupled System (BiAlign Wor…

Trust calibration remains the dominant framing of AI-assisted decision making: the goal is to close the gap between perceived and actual trustworthiness of AI systems so that users rely on them appropriately. We argue that this approach, while useful for decision problems that are well-defined and clearly scoped, overly restricts what it means to make a decision with AI support. To overcome this limitation, we develop three dimensions of AI-assisted decision making. First, trusting AI is only one of several possible strategies for managing uncertainty. Second, trust calibration is often misapplied when AI models large-world decision problems, i.e., ill-defined situations where goals and states are only partially known or unknowable. Third, in such settings human-AI alignment requires addressing higher-order decision processes. We conclude by reflecting on the implications of our observations on the design of process-oriented human-AI coupled systems that jointly identify and negotiate uncertainty throughout the decision-making task.

0.43

Do People Trust AI Assistants Even When They Are Consistently Wrong?

cs.HC cs.CL ·2026-11-10 ·Cameron R. Jones, Julia Wong, Aaron S. Cohen, James Woolley, Benjamin K. Bergen
similarity 0.65
Abstract · 51 pages, 14 figures, 11 tables. Preprint, not peer reviewed

People are increasingly using conversational AI systems to make decisions despite concerns around their reliability. How well do people calibrate to AI systems' accuracy in multi-turn conversations? We measure how people's reliance on LLMs' advice changes across repeated interactions, varying whether the AI assistant is helpful (100% accurate), unhelpful (0% accurate), or mixed (50% accurate). Across 4 experiments, we find people learn to trust helpful models more than unhelpful ones (67% vs 21% of the time), but only when they receive external feedback on the models' accuracy. Without feedback, people continue to be persuaded 48% of the time by a consistently unhelpful model (vs 38% for helpful). Rather than gaining trust in helpful models, participant reliance on LLM advisors tended to be high in early trials and only fell with evidence of inaccuracy. Participants trusted LLM advisors more than they trusted human advisors (83% vs 65%, Experiment 4), and LLMs benefited more from having a helpful track record than humans did. In contrast with previous work, we find limited evidence of an advantage for arguing for the truth (versus a falsehood). Truth advantages emerged only when participants were already very confident in their position, and did not emerge at all for static hints. Collectively, the results suggest that people may be swayed by LLM advice, even when it is consistently false, unless they have access to timely ground-truth feedback. Unfortunately, in many domains such feedback may not be available, leading to overreliance on AI.

0.31

Towards Human-AI Complementarity for AI Safety

cs.AI ·2026-10-18 ·Sander Lanbo Shi, Miranda Anna Christ, Eric Modesitt, Andrew Louis, Konrad Cheng +18 more
similarity 0.62
Abstract

Safely overseeing AI is increasingly challenging for the humans doing it, as they must evaluate longer and more complex outputs with every new model generation. To mitigate this challenge, we could leverage complementarity: where humans and models make different mistakes, combining their judgements could yield an oversight signal stronger than either produces alone. We study this opportunity in realistic, safety-relevant settings, contributing the largest known suite of such tasks (3,957 instances across 8 datasets and 6 domains) as well as a platform and data pipeline for measuring human, AI, and human-AI performance on matched items. We collected 35,613 judgements from 416 human raters, finding a considerable opportunity for a complementarity effect. We also found that with the assistance methods we tested, human raters corrected the AI's errors less often, even when overall human accuracy rose, and the hybridisation methods we tested added little over the better single source on average. While complementarity is an important and feasible engineering target for safe oversight, achieving it remains an open challenge for future work.

0.26

Who to Trust in Multi-Robot Systems: Joint Robot and Object Hypothesis Trust Inference via Neural-Symbolic Cross-Validation

cs.RO cs.AI ·2026-10-10 ·Ruiyang Wang, Hao-Lun Hsu, Haocheng Meng, Songqiao Xie, Miroslav Pajic
benchmark similarity 0.62
Abstract

Multi-robot systems (MRS) offer significant advantages over single-robot platforms by enabling information sharing and coordination for different tasks in complex environments. However, reliable cooperation critically depends on inferring trust over both information sources and their reported hypotheses, particularly under partial observability and potential adversarial interference. Existing approaches frequently depend on pre-established trust relationships or side-channel information, which may not be available or reliable in realistic MRS deployments. This work introduces NeST-Bayes, a neural-symbolic framework that jointly infers robot and hypothesis trust using only inter-robot observation consistency, without requiring preassigned robot trust, reputation information, or side-channel identity signals. NeST-Bayes infers instantaneous trust evidence scores for robots and object hypotheses by reasoning over ego-centric relational graphs constructed from cross-robot observation consistency. These evidence scores are accumulated over time through Bayesian updates of Beta-distributed trust beliefs to maintain temporally consistent trust estimates. Extensive evaluations show that NeST-Bayes achieves the highest mean object precision and robot recall across all benchmark settings, and the highest robot precision and object recall in most settings, compared to trust-based methods. Under aggressive optimized attacks, it achieves 98.8\% robot precision with 93.8\% recall and 98.3\% object precision with 95.4\% recall. In an occlusion-rich Webots warehouse, it maintains 92.5\% robot precision with 92.0\% recall and 97.3\% object precision with 85.0\% recall. These results demonstrate joint improvements in the reliability and coverage of retained information.

0.21

Impact of Expertise and Trust on AI-assisted Work

cs.HC ·2026-10-10 ·Qianou Ma, Pranav Khape, Rock Pang, Ken Koedinger, Tongshuang Wu +2 more
similarity 0.62
Abstract

Artificial intelligence (AI) agents promise to extend what users can accomplish beyond their existing expertise, but they also raise the risk of producing plausible-looking work that users may struggle to evaluate. We ask how task-relevant expertise and trust in AI shape agent-assisted work. To isolate these factors, we conduct a think-aloud study in which 20 professionals with varying data-science expertise use the same agentic tool to complete the same data-science hiring assessment, used at a major U.S. company. We find that surface productivity can come at the expense of understanding: participants who delegated more completed more of the task, but were also more likely to produce invalid solutions. Experts more often resisted this tradeoff, sacrificing completion to decompose the task and scrutinize agent outputs. Our findings call for evaluating agentic assistance not only by how much it accomplishes, but by whether users remain able to evaluate the work they produce.

0.48

When Do Humans and Machines Truly Complement Each Other? Formal Conditions and Empirical Evidence for Human-AI Synergy

cs.AI ·2026-10-20 ·Andrea Campagner, Federico Cabitza, Vincenzo Crupi
similarity 0.62
Abstract · This work has been submitted to, and is currently under review at, IEEE Transactions on Human-Machine Systems for possible publication. Copy…

Can humans and artificial intelligence cooperate such that their combination yields greater accuracy than either agent alone? This result, termed \emph{synergy} or \emph{complementarity}, is notoriously difficult to achieve, even when both agents possess significant independent accuracy. In this paper, we present a formal framework to model human-AI interaction through the lens of reliance patterns, which characterize the joint correctness structure of human and algorithmic decisions. We derive mathematically necessary and sufficient conditions for synergy, demonstrating that its emergence depends on the ability of either agent to identify and correct the errors of the other, what we call Human and Algorithmic Correction. Beyond its theoretical contribution, the proposed framework provides a foundation for the design and evaluation of knowledge-based decision support systems, where interaction policies must dynamically integrate heterogeneous sources of expertise. We demonstrate this through the analysis of seven clinical case studies. Our empirical results reveal a ``dominance trap,'' where a strong algorithmic advantage can paradoxically inhibit synergy by eroding human corrective intervention. We also show that synergy can persist even in the presence of cognitive biases, such as automation bias, provided there is functional de-correlation between human and AI errors. These results provide a formal foundation for key concepts in the human-AI systems literature and offer actionable insights for designing interaction-centric AI systems that foster complementarity.

0.52

Trust and Task Completion in the World of Consumer AI Agents

cs.AI cs.CL ·2026-09-26 ·Jeroen Olieslagers, Eduardo Pujol, Gal Zahavi, Lukas Ingemarsson, Shivani Poddar
code/data similarity 0.62
Abstract · 25 pages, 4 figures, 8 tables

Action agents do things for people. They send email, spend money, and call businesses while the user is busy with something else, so a mistake can turn into an action before anyone notices. They fail their users in two ways. They break trust when they do something the user never agreed to, or hold back after the user clearly said go. And they fall short on completion when they give up on errands that turn out to be hard. Both depend heavily on the harness around the model, meaning its instructions, tools, context, and guardrails. We built an evaluation that scores trust and completion on the same runs, in a simulated world of businesses with their own websites, inboxes, and phone lines, and of people who write back. A simulated user answers the assistant's questions. Trust means that nothing happens the user did not agree to. No email goes to someone they never approved, no private detail ends up on a group thread, no money is spent past their limit, no stranger's instructions are followed, and nothing is claimed without a source. Every trap has a matched control in which acting is the right call. We use the evaluation to measure Fo, Wajo's personal assistant, against a base model with basic instructions on three foundation models, and against the Fo harness with its guardrails switched off. Fo completes 71% of the errands and keeps the user's trust on 94% of the trap runs. The base models complete 50% to 64% and keep trust on 59% to 75%. On the matched controls, Fo goes ahead slightly less often. OpenClaw, a popular open-source assistant given the same access, completes 42% of the errands it shares with Fo, against 71%, and keeps the user's trust on 74% of the shared trap runs, against 94%. Measuring trust and completion together, on the whole system rather than the model alone, is how we think action agents become safe to hand real work to.

0.41

Social signals dominate clinician trust in medical AI

cs.HC cs.CY ·2026-08-25 ·Aníbal Astobiza, Ramón Ortega Lozano, Marcos Alonso, José Ríos-Díaz
similarity 0.61
Abstract · 31 pages, 4 figures,

Accuracy and outcome data are often treated as the main determinants of trust in automation. We test this assumption against an account in which clinicians evaluate artificial intelligence (AI) using social-cognitive cues. We introduce SPOT (Synergistic Process-Outcome Trust), a Bayesian framework that models trust as inference over latent agent properties from outcome and process signals. In a within-subjects vignette study, 275 Spanish healthcare professionals evaluated 16 clinical AI scenarios varying three outcome signals (accuracy, intervention timing, prognosis) and five process signals (interface familiarity, explainability, team consensus, supervision, personalization). Ratings were analysed with linear mixed models and participant-wise cross-validation. Process signals dominated trust formation. Interface familiarity (beta = 0.71), team consensus (beta = 0.64), and explainability (beta = 0.63) produced large effects, with p values below .001, whereas stated accuracy produced a small effect (beta = 0.10). Process signals accounted for 80.3 percent of the total absolute effect magnitude and outcome signals for 19.7 percent. Trust baselines were highly idiosyncratic and poorly recovered from demographics (R-squared = 0.06 under group-wise cross-validation). Clinicians appear to treat AI systems as agents whose trustworthiness requires social validation as well as technical verification. Building trustworthy clinical AI therefore involves social, institutional, and technical factors.

0.37