CS Paper Radar

When Does the Public Become Suspicious of Bots? Demand-Side Evidence from Botometer Query Logs

Tuğrulcan Elmas

cs.CY cs.SI ·submitted 2026-09-17 16:39 UTC · v1

Abstract

We study private bot-checking behavior from the demand side: when people suspect an account is automated, whom they suspect, and what follows. Using Botometer's server-side query logs, the most widely used bot-detection service, we treat each query as a behavioural trace of suspicion. We analyze over 1 million public checks of Twitter accounts from 2020 to 2023, enriched with 3.2 billion tweets from the contemporaneous 1% public stream. Collective suspicion spikes with platform crises, most sharply around the 2022 Musk-Twitter bot dispute. Checked accounts are older and more prolific, have more followers, and post promotional, political, and crypto content. Accounts that draw collective suspicion have higher bot scores and are more likely to be suspended. Bot-related public attention and Botometer activity are elevated during the same broad periods, although their short-run fluctuations are largely uncoupled. Public feedback focuses on first-person identity claims for humans, and evidence-based arguments citing posting rate, political content, and cross-account coordination for bots and cyborgs. Bot suspicion thus constitutes a mass, distributed form of platform auditing that tracks meaningful signals of automation, establishing audit-tool query logs as a novel lens on public responses to platform manipulation.

Closest papers

Nearest neighbours by meaning across the whole index.

Social bots weaken activist cohesion

cs.SI cs.CY ·2026-08-31 ·Linda Li, Orsolya Vasarhelyi, Balazs Vedres
similarity 0.68
Abstract · 4 figures, 39 pages (including supplementary information)

Social bots now make up a substantial share of online political communication, where they are studied mainly as producers of misinformation and amplified content. Far less is known about whether their presence reshapes the human relationships that hold movements together. We ask whether exposure to bots during a protest peak is followed by the erosion of cohesion in human networks. Tracking retweet networks of core participants in the 2020 Black Lives Matter (BLM) protests before, during, and after the peak, we measure change in cohesion at two scales: triadic closure in individual ego networks and edge density within detected communities. Greater bot exposure during the peak predicts steeper subsequent declines in human cohesion at both scales, and the loss concentrates among supporters of the movement. Bots may weaken activism less by changing what people believe than by dissolving the ties through which collective action is sustained.

0.22

Proof-of-Agentness: Towards Verifying Agents on the Web

cs.AI ·2026-10-26 ·Zuyao Xu, Xiang Li
similarity 0.66
Abstract · 12 pages, 7 figures

CAPTCHAs and bot detection have helped protect Web services from automated abuse for decades by distinguishing humans from bots. However, the emergence of LLM-powered agents introduces a new setting: Web services increasingly accommodate agents as legitimate users. Without a way to distinguish agents from other automated clients such as simple bots and scripts, these services risk large-scale automated junk submissions that waste service resources and consume users' attention or tokens. In this paper, we investigate a novel question for today's Web: "How can a Web service obtain evidence that a client is an agent?" We introduce the concept of "Proof-of-Agentness" (PoA), a mechanism for distinguishing agents from other automated clients, motivated by the parallel between human attention and an agent's inference resources. We outline two approaches: active verification through challenge-response and passive analysis of service-visible signals. Eight language models achieve an overall accuracy of 96.5% on our challenges, with an average overhead of 1,647 tokens. For passive analysis, we conduct an IRB-approved user study comparing the Web interactions of twelve human participants and representative Bot and Agent implementations, achieving an average classification accuracy of 95.8% across four classifiers, motivating further study.

0.29

Crude, Commercial, and Self-Referential: Chinese-Language Coordinated Activity in Japanese-Language X

cs.SI cs.CY ·2026-09-28 ·Kei Ichikawa, Bruno T. Sugano, Genta Toya, Wu Qianyun, Yasuhiro Hashimoto +3 more
similarity 0.66
Abstract

Malicious coordination has long been regarded as a principal source of information ecosystem pollution. Here, we focus on crude, text-repetition-based coordination. As the demand for mitigating its dissemination has grown, scholars have studied such coordination, focusing especially on bot detection. Few studies, however, have characterized malicious coordination per se or examined how it elicits reactions from general users. Leveraging a dataset of 734,173 Chinese-language coordinated accounts and around 495 million coordinated posts published between May 2024 and March 2026, this study analyzes the characteristics of coordinated behavior and how general users react to coordinated posts. We report three findings: (1) most coordinated accounts are crude and retain the classic marks of automation, and the same criterion applied to Japanese-language accounts over the same month yields a share six times lower; (2) their content is overwhelmingly non-political; (3) regarding their reach, most reactions within large observable cascades originate from coordinated accounts themselves, while posts classified as potentially harmful or illegal material receive a comparatively high proportion of reactions from outside the Chinese-dominant population. We provide a longitudinal quantitative map of crude Chinese-language coordination appearing in X's Japanese-classified stream.

0.12

Measuring Users' Security and Privacy Concerns About Third-Party Community Bots on Discord

cs.CR cs.HC ·2026-10-27 ·Kofi Ampomah, Tu Le
top venuecode/data similarity 0.65
Abstract · Accepted to NDSS Symposium 2027. 20 pages. Artifact: https://doi.org/10.5281/zenodo.23583221; code: https://github.com/ua-spectral/BotLens

Third-party community-based bots are central to how social platforms such as Discord work. They moderate content, manage roles, play music, and support AI-assisted conversations across millions of servers. Yet when a bot is installed in a server, it can access messages, user profiles, and server metadata concerning members who did not participate in the installation decision, with no standardized way to communicate the associated security and privacy implications. In this paper, we analyze 23,102 third-party Discord bots and 183,142 user reviews spanning 49 detected languages and five years of feedback (2021-2026). We employ LLM-based knowledge distillation and fine-tune a DeBERTa-based classifier to identify candidate security and privacy (S&P)-relevant reviews. After manually validating the complete classifier-positive set, we retain 8,832 reviews identified as S&P-relevant, representing an observed pipeline yield of 4.82%, and map them to a 31-theme S&P taxonomy extending prior work. Our results show that S&P-related reviews focus heavily on Data Sharing (30.5%) and Data Collection (16.2%). Confession and anonymity bots have pipeline-identified SPRRs above 50%, more than ten times the ecosystem-wide pipeline yield of 4.82%. Star ratings may obscure the frequency and valence of S&P-related discussion, while conventional metadata features, such as permission count and administrator status, show only weak associations with user-reported S&P concerns. We then build BotLens, a proof-of-concept auditing framework that combines review-derived S&P signals with bot metadata to provide on-demand decision support for the community-based bot ecosystem. BotLens exposes six tools through a Model Context Protocol server, supporting bot-level checkups, server-wide comparison, and integration with AI agents.

0.63

Birds of a Feather Flock Together: Network-Based Detection of Coordinated Disinformation Campaigns on Telegram

cs.SI cs.CY ·2026-09-22 ·Panteleimon Tsagkarakis, Emmanouil Papadogiannakis, Evangelos Markatos
published similarity 0.65
Abstract

In recent years, disinformation has increasingly proliferated across social networks. The firing of fact-checkers from Meta and the disbanding of Twitter's Trust and Safety Council suggest that this trend will continue to escalate. While disinformation sources (e.g., social network accounts) can sometimes be identified, new accounts emerge daily, making tracking a moving target. In this work, we propose a graph-based methodology to discover previously unknown Telegram accounts that spread disinformation. Starting from a set of verified disinformation groups, we examine their interconnections and identify new accounts that contribute to the dissemination of false narratives. Our approach is language-agnostic, as it relies solely on structural relationships between accounts rather than analyzing their message content. Using this approach, we identify 37 previously unknown disinformation channels (a threefold increase). We demonstrate that Telegram channels display extremely dogmatic behavior with up to 80% of messages being labeled as propaganda. Our findings reveal that misinformation on Telegram spreads within tightly interconnected clusters, in some cases, with over 86K identical messages being shared to multiple channels, suggesting coordinated disinformation campaigns.

0.38

Breaking and Defending LLM-Powered Social Media Bot Detection Systems

cs.AI ·2026-08-16 ·Nof Orenstein, Yoni Birman
published similarity 0.64
Abstract · Accepted at ACISP 2026 (Australasian Conference on Information Security and Privacy). Also accepted as a poster at IEEE Symposium on Securit…

The rise of social media bots poses a persistent threat, enabling misinformation, opinion manipulation, and the erosion of trust in online platforms. To combat this, machine learning systems have been developed to detect and limit bot activity, but attackers continuously adapt through techniques such as adversarial learning and behavior imitation, fueling an ongoing arms race between bots and detection tools. Recent advances in large language models (LLMs) have significantly improved bot detection by enabling deeper semantic and contextual analysis of accounts and their content. However, this shift also introduces new attack surfaces, allowing adversaries to craft exploits that directly target the reasoning and generation mechanisms of LLM-based classifiers. Industry tools such as Anthropic's Claude Code Security similarly leverage LLMs for security-critical decisions, further motivating a careful study of their attack surfaces. In this work, we investigate both the offensive and defensive aspects of LLM-powered, threat-specific cybersecurity applications. While centered on the challenge of social media bot detection, our methodology and insights generalize to a broad class of LLM-powered cybersecurity systems, including phishing detection, email classification, and fraud analysis. We introduce two novel adversarial attack strategies that systematically exploit the semantic and contextual weaknesses of LLM-based classifiers, degrading their detection accuracy by up to 48%. To counter these threats, we propose a robust multi-LLM defense architecture designed to preserve detection reliability under adaptive adversarial conditions. Our solution, LSABRE (LLM-powered Social Adversarial Bot Recognition Ensemble), is a multi-LLM framework that substantially improves robustness across a range of attacks, maintaining 86% detection accuracy even under strong, adaptive adversarial pressure.

0.40

Social bot detection in the age of ChatGPT: Challenges and opportunities

cs.CY cs.CL ·2026-10-01 ·Emilio Ferrara
publishedsurveyhighly cited similarity 0.64
Abstract

We present a comprehensive overview of the challenges and opportunities in social bot detection in the context of the rise of sophisticated AI-based chatbots. By examining the state of the art in social bot detection techniques and the more salient real-world application to date, we identify gaps and emerging trends in the field, with a focus on addressing the unique challenges posed by AI-generated conversations and behaviors. We suggest potentially promising opportunities and research directions in social bot detection, including (i) the use of generative agents for synthetic data generation, testing and evaluation; (ii) the need for multimodal and cross-platform detection based on network and behavioral signatures of coordination and influence; (iii) the opportunity to extend bot detection to non-English and low-resource language settings; and, (iv) the room for development of collaborative, federated learning detection models that can help facilitate cooperation between different organizations and platforms while preserving user privacy.

0.48

Information Operations Exploit APIs to Manipulate Social Media

cs.CY ·2026-10-01 ·Manita Pote, Alessandro Flammini, Filippo Menczer
similarity 0.62
Abstract

Research on information operations has focused primarily on the coordinated accounts involved in such campaigns and in the content they promote. Bad actors manage the activity of inauthentic accounts through programmatic interfaces (APIs), but the role of this underlying infrastructure in enabling and uncovering such coordinated behaviors remains largely unexplored. This study investigates how the Twitter (now X) API was used to coordinate influence operations on the platform. Since API access is managed through developer apps, we extract app metadata from 43 operations taken down between 2018 and 2021 and apply three complementary computational approaches to study diverse forms of manipulation. Our analysis reveals several recurring patterns: API-as-a-service infrastructure reused across campaigns, gaming apps performing unauthorized engagement without user consent, spoofed app names to hide activity origins, and synchronized app networks indicating centralized control. Adding app usage as a coordination indicator provides a modest but consistent improvement in detection. These findings suggest that social media APIs play an important role in enabling online manipulation, and that the removal of app-level metadata by Twitter/X has eliminated an important tool for platform transparency and moderation.

0.17