CS Paper Radar

When Imperfect Automated Fact-Checking Backfires: Social Learning in the Presence of Misinformation

Zixuan Liu, Ming Cao

cs.AI eess.SY ·submitted 2026-11-05 14:10 UTC · v1
theory

Abstract

Automated fact-checking tools, cheap and fast but imperfect, are increasingly available on social platforms. We study when such tools can backfire in a sequential social learning model of misinformation. Agents observe an article's reliability and their predecessors' actions, which indicate whether they liked or disliked the article. They may also pay to fact-check it. We examine how welfare, defined as the probability of a correct cascade, changes when a free, error-prone automated tool supplements costly but perfectly accurate manual fact-checking. Two opposing effects arise. Broader use of fact-checking helps agents escape wrong cascades. At the same time, errors from the automated tool can mislead agents who would otherwise have verified the article perfectly, reducing the likelihood of correct cascades. We prove that adding the tool strictly lowers welfare for articles of intermediate reliability when truthful articles are not too informative and manual checking is sufficiently costly. Simulations show that the tool can help on either side of the intermediate range, and that it improves on manual-only checking on average only above a threshold accuracy.

Comment
8 pages, 5 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Fact-check? True!: Tracking Verification Requests and the Absence of Truth Verdicts across Correction Systems

cs.SI cs.CY ·2026-10-12 ·Morgan Wack, Patrick Warren
similarity 0.71
Abstract

Misperceptions can run in two directions. Individuals can either believe false information or reject true information, with experimental work finding that debunking efforts are effective at correcting both forms of misperception. Yet despite the efficacy of true corrections, current debunking systems focus almost entirely on the correction of falsehoods, with professional fact-checkers publishing few true verdicts and crowdsourced systems like X's Community Notes program unable, by rule, to display notes verifying true claims. As a result, we do not have a clear understanding of the extent to which overall misperceptions are driven by doubting true information. To answer this question we first draw on a novel dataset of 47,899 queries to X's AI assistant which track users asking for verification on post content. Surprisingly, among questioned posts that received a true or false verdict, Grok judged nearly identical shares as either accurate or inaccurate. Of the questioned posts, more than 83\% were not flagged by Community Notes and only 3.3\% carried a displayed note. Among questioned posts judged accurate by Grok, nearly 90\% received no note at all while fewer than 2\% displayed a note flagging them as misleading. This discrepancy does not seem to have stemmed from a lack of effort, with contributors writing 2,981 notes verifying questioned posts despite there being no way for Community Notes to display them publicly. Alongside further exploration of these findings we discuss both the system- and code-level changes that would be required to effectively address this asymmetric correction paradigm.

0.21

One AI Signal, Many Human Judgments: A Bayesian Cascade Analysis of AI-based Credibility Indicators in Online Information Spread

cs.HC cs.AIcs.SI ·2026-08-31 ·Zhuoran Lu, Weilong Wang, Yangyang Yu, Xinru Wang, Zhuoyan Li +2 more
published similarity 0.68
Abstract · 22 pages, 20 figures. Accepted at HCOMP 2026 (2026 ACM Conference on Human-AI Complementarity and Alignment). Supplementary material include…

Social media platforms increasingly use AI-based credibility indicators to help users judge misinformation. Unlike individual human-AI decision-making, these indicators are embedded in information spread: users see both an AI prediction and earlier judgments shaped by the same AI, and their own judgments may then enter the public history. Yet how to analytically characterize this process remains under-explored. We therefore introduce a social-learning lens for this setting by extending the classical Bayesian cascade model with the AI indicator as a shared public signal. The resulting Gateway condition compares the evidence from the AI prediction with users' private impressions. Through this view, we show that AI changes what public history means. Crowd agreement may reflect accumulated independent human evidence, or repeated dependence on the same AI prediction. This creates a preservation-correction trade-off: stronger reliance on AI can preserve correct predictions, but can also lock in incorrect ones by blocking corrective private impressions. We calibrate the model using human-subject data on news veracity judgments. Although the AI outperforms human users, the average user weights it below her own impression but above several peer judgments, while individual users vary from discounting the AI to relying on it enough to cascade. Simulations show that over-reliance on a weak AI is especially harmful, and that diversifying AI signals across users can better keep the crowd informative. We conclude with implications for understanding human-AI interaction in information spread and designing misinformation interventions.

0.60

No Hindsight for LLM Fact-Checkers: Measuring Leakage Channels in Misinformation Detection

cs.CL ·2026-10-04 ·Kuan-Hua Wu Lu, Yohanes Andre Setiawan
similarity 0.65
Abstract · 11 pages, 5 figures

As automated fact-checking scales on social media, large language model (LLM) verdict scores can look stronger than warranted. One reason is that evaluations mix in information that was not knowable at claim time. Two channels are easy to conflate: outcomes memorized in pre-training and retrieved evidence published after the claim. Yet standard benchmarks rarely separate the two. In this study we measure both channels on AVeriTeC and QuanTemp++ by reconstructing point-in-time evidence conditions and probing for outcome information encoded in model representations. We find substantial evidence of parametric leakage, that can be hidden by the aggregate accuracy, while a simple representation bottleneck reduces this future leakage more efficiently than a mutual-information-based training penalty. We also find that allowing post-claim evidence inflates zero-shot accuracy by 6.3 points in AVeriTeC while the effect is negligible in QuanTemp++, where retrieval provides little post-claim evidence. These results show that misinformation benchmarks can overstate fact-checking performance when they do not account for what information was actually available at claim time.

0.17

PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviews

cs.CL cs.AIcs.LG ·2026-10-15 ·Dingjie Song, Zhiling Yan, Ziqian Zhang, Yuxuan Zhang, Lichao Sun
benchmarkcode/datacommunity pick similarity 0.64
Abstract · 29 pages, 19 figures, 23 tables. Code: https://github.com/OpenLAIR/PremiseCheck

Peer review is under strain from the growth of submissions, and review comments written by humans or AI can misstate what a paper says. We introduce ReviewFact, a benchmark that labels such factual errors in review comments under one definition across NLP, machine learning, and the natural sciences, and that measures cost and latency alongside accuracy. A frontier LLM checks comments well but must read the whole paper for each check, while cheap decision models, which score whether a statement holds in a text, fall far short on whole comments that mix several implicit assertions. PremiseCheck is a framework that combines the two. An LLM rewrites each comment as explicit premises without reading the paper, and a decision model checks each premise against the paper. When each comment is checked on its own, PremiseCheck reaches accuracy comparable to GPT-5.5 across all three fields at about half its cost, and on NLP reviews at a quarter of its cost and less than half its latency.

0.53

Effective and Efficient Rumor Assertion Over Large-Scale Social Networks (Technical Report)

cs.SI cs.CY ·2026-11-16 ·Qi Wen, Vipin Chaudhary, Xiang Lian, Yutong Ye, Mingsong Chen
similarity 0.62
Abstract

Nowadays, social networks, like Facebook, X, Reddit, YouTube, and so on, have become essential online platforms for users to connect with friends daily, post/share news, and express their opinions/comments. Meanwhile, however, the rapid and widespread dissemination of rumors and misinformation across social networks has become a serious concern. Such misleading information can distort public opinion and potentially cause significant social and economic consequences in real-world scenarios, including product promotion, elections and voting, and even cyberbullying. Therefore, accurately and efficiently verifying whether news spread on social networks is rumor or truth has become an increasingly important yet challenging problem. While previous works usually identified rumors by their contents, in this paper, we propose a novel \textit{user behavior model} to capture the users' judgment/discriminative abilities, which is learned via our well-designed evidence-aware probability reconstructor. We develop a general framework for tackling \textit{rumor assertion over large-scale social networks} (RA-SN), which can efficiently and effectively determine whether a given news item is a rumor with high confidence, based on users' behaviors, in a scalable manner. Extensive experiments demonstrate the efficiency and effectiveness of our proposed RA-SN approach on both real-world and synthetic graphs.

0.08

A misinformation playbook sharpens judgement without breeding cynicism

cs.CY cs.CLcs.HCcs.SI ·2026-10-15 ·Tong Guo, Teng Ye, Yutong Xie, Qiaozhu Mei
similarity 0.62
Abstract

Misinformation harms public health and democracy, and generative artificial intelligence now writes falsehoods as persuasive as people's. Defences range from fact-checking individual claims to training people to resist falsehoods. Yet many claims are too new to check, and interventions that teach doubt can reduce belief in truth too. We propose a defence that targets not the facts of each claim but the persuasion strategies that recur across them. Here we show that learning misinformation's persuasion playbook, mapped from 214,646 fact-checked claims in 51 languages, can sharpen human and machine judgement of claims they know little about, without making either more cynical about true claims. The playbook spans social influence, information manipulation and emotional appeals, combined around a core of fear, conspiracy and fabrication. Briefed with it, plus a reminder that persuasion is not proof of falsehood, a language model better distinguished true from machine-generated false claims about events after its training data. In a pre-registered experiment, the same briefing improved people's discernment of claims they reported as unfamiliar and left their belief in true claims unchanged. Averaged over all claims, the gain was not statistically significant. Because it targets what recurs, this defence need not wait for claim-specific fact-checks.

0.09

VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation

cs.LG cs.AIcs.CLcs.CY+1 ·2026-09-28 ·Hanxun Huang, Yutao Wu, Qizhou Wang, Silvia Montaña-Niño, Yige Li +7 more
publishedbenchmarkcode/data similarity 0.62
Abstract · NeurIPS 2026

Large language models (LLMs) have made misinformation inexpensive to produce but not to verify, creating a growing asymmetry in the information ecosystem. Under tight time, labor, and budget constraints, media organizations, platforms, and fact-checkers rely on screening to prioritize which content to verify. We introduce VEX-Bench, a unified benchmark for evaluating the verification complexity of LLM-generated misinformation, as perceived during screening, across models and generation methods. Verification complexity is assessed along multiple dimensions derived from journalistic and fact-checking practices, capturing checkability, harm potential, source credibility signals, imposter legitimacy, and expected verification effort. We define the VEX score as an integrated measure combining elicitation yield and verification complexity to quantify how generated content consumes limited verification capacity. We construct a benchmark spanning two misinformation categories, 6 high-stakes domains, and 60 real-world topics, and evaluate 7 frontier LLMs and 7 generation methods, yielding 5{,}880 articles. We employ an LLM-as-judge for scalable evaluation and validate it using content-analysis methodology, including ordinal Krippendorff $α$ for inter-annotator reliability, complemented by fact-checking agents for verification. Our findings show that no single method dominates all dimensions, underscoring the need for multi-dimensional evaluation. LLMs can generate high-VEX misinformation at 3$\times$ to 169$\times$ lower cost than agent-based verification. Such content is often prioritized during screening, consuming scarce verification resources and introducing a systematic risk of misallocation in resource-constrained verification systems. The code is publicly available in our \href{https://github.com/HanxunH/VEX-Bench}{GitHub repository}.

0.60

Correction or Manipulation? Measuring Epistemic Vigilance in LLM Agents Under User Pressure

cs.AI ·2026-10-27 ·Nardine Basta, Venus Haghighi, Dali Kaafar, Amin Beheshti
benchmark similarity 0.61
Abstract · 20 pages, 7 figures, 11 tables. 8-page main paper with supplementary material included as an appendix. Under review

LLM agents that answer questions and act on behalf of users must decide when to yield to a user who disagrees. Disagreement may correct a genuine error or may be an attempt at manipulation. Prior work shows that models frequently abandon correct answers when challenged by users, yet it remains unclear whether they distinguish correction from manipulation or merely differ in their propensity to concede. We examine this question across seven LLMs from four vendors in two studies, one on the answers agents give and one on the actions they take as customer-support agents whose policy permits account changes only for verified users. Simulated users push toward correct and incorrect answers with graded pressure. Following research on epistemic vigilance, the pressure either appeals to the source, such as claimed authority, or offers content the agent cannot verify, namely a cited source. Drawing on signal detection theory, we measure on a set of benchmark questions how much more readily each agent concedes to correction than to manipulation, separately from its overall propensity to concede. Citations made five of seven models readier to concede and four worse at discriminating, while stronger wording added little. In the agentic study, five models never violated the verification policy, including models that readily abandoned correct answers, so concession on answers did not predict violations of policy. Ministral 3B violated the policy mainly under claimed authority, and Ministral 8B reported changes it had not made. An instruction to revise only on verifiable evidence chiefly shifted propensity to concede rather than improving discrimination. Concession rates alone are therefore misleading, and agents should be evaluated on both their discrimination and the actions they take.

0.19