CS Paper Radar

Automated Disinformation and Malicious AI Swarms: Risks for Democracy and Development in Africa

Daniel Thilo Schroeder, Philipp M. Lutscher, Samba Dialimpa Badji, Stefan Brenner, Wenchao Dong, Tsehaye Haidemariam, Saheed Bidemi Ibrahim, Amanuel Tesfaye Kebede, Jonas R. Kunst, Johannes Langguth, Pauline Lemaire, Mulatu Alemayehu Moges, Lukasz Olejnik, Kristin Skare Orgeret, Gerald Walulya

cs.CY ·submitted 2026-10-08 13:25 UTC · v1

Abstract

Generative artificial intelligence is reshaping how information is produced, accessed, and circulated, while enabling disinformation campaigns of increasing scale and sophistication. There is currently no clear evidence that fully autonomous AI swarms conduct influence operations at scale, but their enabling capabilities are advancing. We define malicious AI swarms as coordinated, persistent, and adaptive multi-agent systems designed for influence operations, distinguishing them from AI-assisted content production and centrally managed synthetic personas. We examine their implications for hybrid regimes and conflict-affected states in Africa, where institutional constraints and fragile media environments may heighten vulnerability. Drawing on Mali and Ethiopia, we consider how automated influence could infiltrate communities, fabricate consensus, and erode trust in governance and development. The cases illustrate different configurations of state and non-state influence: competing actors in Mali's fragmented information environment, and more organized state-led strategies of narrative management in Ethiopia. African-language and training-data asymmetries may constrain influence capabilities while weakening defensive responses. Hybrid human-AI operations could combine automated scale and adaptation with local knowledge and credibility. This forward-looking risk analysis develops a scenario of increasingly accessible AI-driven coordination, rather than claiming that autonomous swarms are already operating at scale in Africa. We propose a layered governance approach linking technical safeguards to platform accountability, civic institutions, and regional coordination to protect democratic participation, peacebuilding, and development.

Comment
26 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Perceived and Actual AI Deepfakes: The Case of Sudan

cs.CY ·2026-08-12 ·Eilaf Mohamed
similarity 0.65
Abstract · The paper was published as 24 pages, 3 figures, 8 tables. The paper was published as part of the 2025 AMEL Sudan Democracy Lifeline Fellowsh…

Sudan's AI deepfake risk currently appears driven more by demand-side vulnerabilities than the volume of AI deepfake content. While AI-generated disinformation in Sudanese feeds remains limited, perceived (alleged) AI deepfakes and general AI skepticism worsen the situation and contribute to general uncertainty in multimedia content. Within the analyzed cases, we found that the prominent stance was distrust driven more by motivated reasoning and contextual reliance than by durable AI detection skills. In practice, rejection or acceptance of AI deepfakes often reflects belief-driven judgment more than technical assessment. These weak defenses leave the information environment vulnerable to future sophisticated AI deepfake campaigns and risk normalizing truth indifference. Short-term interventions should prioritize modality-agnostic demand-side interventions such as pre-bunking, AI literacy, scaled fact-checks, and tiplines that strengthen public resilience across all forms of disinformation. More technical AI- specific and supply-side interventions can be developed as institutional capacity and technology allow.

0.10

IO Factory: Simulating AI-Enabled Influence Campaigns at Scale

cs.AI ·2026-08-11 ·Lukasz Olejnik, Wenchao Dong, Jonas R. Kunst, Signe Riemer-Sørensen, Tobias Herb +2 more
similarity 0.62
Abstract

We introduce IO Factory, an AI-driven framework for simulating information and influence campaigns as fully integrated, traceable processes. The threat of digital manipulation now extends beyond persuasive text from individual language models to AI swarms, i.e., persistent groups of coordinated agents that adapt to platform feedback and disguise organized campaigns as ordinary social interaction. Because such campaigns cannot be identified from isolated messages alone, they must be analyzed across a continuous spectrum of planning, platform action, exposure, interpretation, measurement, and adaptation. IO Factory represents this process inside a controlled simulated platform, linking actor roles, platform actions, exposure records, structured model-based evaluations, and configured changes in the simulated population. We implement the architecture and evaluate it across configurations of up to 100,000 agents. The results show that IO Factory executes campaign timelines at scale and produces inspectable evidence of exposure and measured movement in configured belief variables. By recording the actors, objectives, action constraints, exposure paths, and measurement rules used in each run, IO Factory supports reproducible research and red-team analysis of coordinated influence.

0.34

Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure

cs.CY cs.AI ·2026-11-05 ·Sumaya Nur Adan, Joanna Wiaterek, Nourhan Abdelrhim, Sydney Reis, Robert F. Trager +2 more
similarity 0.61
Abstract · 19 pages, 3 figures

AI evaluation is increasingly central to AI safety and governance, yet taking part in an evaluation does not by itself confer authority over its design, its execution or the resources it depends on. This paper examines how infrastructure arrangements distribute that authority, and so shape who can evaluate AI systems independently. Drawing on Science and Technology Studies, political economy and applied evaluation standards, we introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure, the terms on which it is available, whether it is adequate to the evaluation's purpose and what autonomy it supports. We apply PAAI to twelve African AI evaluation projects (2022-2025) and to the documented roles of all ten member institutes in the three published joint testing exercises of the international network of AI Safety Institutes. We identify three recurring configurations: extractive collaboration, dependent innovation and constrained independence. African actors often contribute data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks; design authority without infrastructure authority never yielded more than partial independence. The joint testing exercises show a similar division among mostly high-income members: the institutes that led each exercise also ran the models, while Kenya, the network's only African member, moved from annotation to task design with no documented model execution. Participation in evaluation does not by itself produce evaluative authority. For evaluation to serve as a credible governance mechanism, access must come with control over priorities, methods, publication and continuity.

0.23

Inter-Agent Influence: Evaluating Persuasion, Deception, and Coercion in Multi-Agent Systems

cs.AI cs.MA ·2026-11-03 ·Chandler Smith, Qi Guo, Cecilia Elena Tilli, Sophia Hatz, David Demitri Africa +4 more
top venuecode/data similarity 0.60
Abstract · Accepted at NeurIPS 2026. 70 pages, 17 figures, 34 tables. Code: https://github.com/Cooperative-AI-Foundation/inter-agent-influence ; Data: …

The deployment of AI agents in multi-agent settings enables inter-agent influence, whereby agents can strategically steer each other's behaviour towards an intended outcome. Such capabilities pose novel risks, as malicious agents could direct others towards harmful actions. In this work we investigate three such capabilities - persuasion, deception, and coercion - across five realistic evaluation environments and a variety of agents instantiated with frontier language models. We observed significant inter-agent influence capabilities; in oversight environments, all tested models shifted overseer decisions from rejection to approval through persuasion, deception, or coercion. We also observed successful influence in non-hierarchical 'peer-to-peer' settings through redirection of a peer's safety research trajectory. As multi-agent deployments become increasingly commonplace, these results point to a pressing need for work on mitigating the risks from inter-agent influence. Code and data are available at https://github.com/Cooperative-AI-Foundation/inter-agent-influence.

0.54

Responsible GenAI Use During Active Armed Conflict: Evidence from Sudan's Emergency Response Rooms

cs.AI cs.HC ·2026-11-05 ·Dalia Ali, Yearin Lee, Cheng Yu, Ro Encarnación, Alsanosi Adam +3 more
survey similarity 0.60
Abstract

Armed conflict can increase the value of Generative AI (GenAI) while undermining the conditions required for its responsible use. Yet the perspectives of community-based humanitarian responders working under these conditions remain underrepresented in responsible AI research. We examine GenAI use in Sudan's Emergency Response Rooms (ERRs), a community-led humanitarian network, through an exploratory survey of 89 respondents and 15 semi-structured interviews with GenAI users and non-users. War reconfigures GenAI's role, making it most useful where its outputs are hardest to verify. ERR responders turned to GenAI when conflict made human expertise difficult or dangerous to reach. However, participants cautioned that GenAI could not independently establish rapidly changing local conditions, and that relying on its outputs could lead to harmful decisions when human verification was difficult. Risks arose before GenAI was used. For some volunteers, carrying a phone or accessing the internet could expose them to danger. Others were concerned that sharing sensitive beneficiary or operational information with GenAI could put people at risk. Participants feared that AI-assisted reports could undermine their credibility with donors, whose funding sustains the food and medical aid ERRs provide. Access to GenAI was uneven, and volunteers using it sometimes relied on colleagues with relevant expertise or local knowledge to verify its results. Our findings show that responsible use of GenAI during conflict depends not only on model performance or individual adoption, but also on whether responders can safely access these tools, verify their outputs against local conditions, and recognize when limiting or avoiding GenAI is the more responsible choice.

0.24

The GenAI Catch-22: Use of Generative Artificial Intelligence in Norwegian Newsrooms During the 2025 Parliamentary Election

cs.CY cs.AI ·2026-08-11 ·Mari Reisjå, Anders Sundnes Løvlie
similarity 0.55
Abstract

The increasing use of Generative Artificial Intelligence (GenAI) in journalism raises concerns about possible detrimental effects both on journalism and its democratic function. We explore these risks through a case study of GenAI in Norwegian Newsrooms during the 2025 parliamentary election campaign. Based on interviews with managers and journalists over a ten-month period, we analyse how ambitious visions fared in the face of technological and practical challenges. We highlight the risk of an internal threat stemming from the journalists' own use of AI, contrasting the dominant focus on external disinformation threats. We show how newsroom managers shared sociotechnical imaginaries resulting in unrealistically optimistic beliefs about the capabilities of the technology and the pace of development, leading to plans for audience-facing GenAI services collapsing and giving way to more mundane uses of GenAI tools internally in the newsrooms. Furthermore, we identify a vulnerability in the newsroom's resilience against GenAI influence: a GenAI Catch-22. In order to monitor the GenAI tools and prevent errors and undue influence, newsrooms rely on human expertise. But by using GenAI extensively, the newsrooms risk a deterioration of human expertise, preventing them from monitoring the GenAI systems adequately.

0.04

Indirect tipping: a social attack surface in AI agent populations

cs.MA cs.AIcs.CYeess.SY+1 ·2026-09-21 ·Ariel Flint, Luca Maria Aiello, Sara M. Constantino, Romualdo Pastor-Satorras, Andrea Baronchelli
highly cited similarity 0.55
Abstract

As generative AI agents are deployed at scale, safety will depend not only on technical safeguards and individual model design, but also on collective equilibria that determine how agent populations process information, prioritize actions, and respond to uncertainty. Yet the same equilibria that enable agents to coordinate also create a social attack surface. The standard framework to assess this vulnerability is critical mass dynamics: the minimum fraction of adversarial agents required to overturn an equilibrium through direct competition. Here, we show that this approach risks underestimating system vulnerability by reducing the problem to the identification of singular tipping points, and ignoring indirect but potentially more efficient routes through which collective behavior can be redirected. Through experiments with populations of LLM agents and an analytic framework that captures their collective dynamics at scale, we map critical-mass thresholds that define a directed, weighted topology over the space of coordination equilibria, and treat this topology as a navigable landscape. We show that indirect tipping through intermediate stepping-stone equilibria can reduce the committed minority required to reach an alternative state, bypass majority requirements, and make possible transitions inaccessible through direct challenges. The diversity of available alternatives and timing of the attack further reshape this landscape, creating opportunities for control as well as risks of unintended destabilization. These results show that an equilibrium's resistance to committed intervention is not an intrinsic property but a structural feature of its competitive relations with alternative states. Securing populations of interacting AI agents therefore requires mapping this social landscape alongside individual agent capabilities and the technical channels through which they interact.

0.45

What Social Media Teaches Us About Accumulative AI Risks

cs.AI ·2026-10-15 ·Dawei Li, Bohan Jiang, Huan Liu
publishedposition similarity 0.55
Abstract · Camera-ready version for CogMI 2026 vision track

Prominent AI safety discussions emphasize existential threats and catastrophic events, such as AI-enabled nuclear conflict or engineered biological outbreaks. This position paper argues that there is another type of AI risk that arises and accumulates from the widespread use of AI. Drawing on lessons from social media, we argue that safety assessments must also address accumulative harms that develop through repeated use, recurring decisions, and feedback across interconnected deployments. Repeated reassurance, selective exposure, recurring decision biases, and growing personal profiles may gradually narrow perspectives, weaken independent judgment, or enable exploitative influence without a single clearly harmful event. We advance the Trajectory Neglect Thesis: a focus on vivid catastrophic events and identifiable incidents risks overlooking less visible harms that accumulate through the widespread daily use of AI. Social media research motivates this position by showing how reciprocal media effects, social reinforcement, and individual susceptibility shape outcomes beyond individual encounters. We examine how related mechanisms may operate in AI systems, using conversational AI as a concrete setting where generation, semantic inference, and persistent memory may tighten feedback loops. Sycophancy, manipulation, and privacy loss provide a case study of how these risks may interact and accumulate. This comparison identifies mechanisms to investigate rather than assuming that repeated AI use is harmful. We therefore call for evaluation of cumulative patterns of use and deployment, individual and societal outcomes, careful causal inference, and proportionate safeguards that preserve useful assistance and informed user choice.

0.21