CS Paper Radar

Cooperation Without Adaptation: LLM Agents in a Collective-Risk Dilemma

Trung-Kiet Huynh, Dao-Sy Duy-Minh, Chi-Nguyen Tran, Phu-Hoa Pham, Phu-Quy Nguyen-Lam, Le Hong Trang, The Anh Han

cs.GT cs.AIcs.CLcs.MA ·submitted 2026-10-16 03:38 UTC · v1

Abstract

Language-model agents are increasingly deployed to act on behalf of people in settings where individual decisions shape shared outcomes. We examine whether five production models adapt their contributions to changing risks and partner behaviour, or instead follow a relatively fixed strategy. In a collective-risk dilemma game (CRG), six players contribute to a shared pool over ten rounds; if the pool falls short of its target, a catastrophe may destroy their remaining resources. Four models contributed approximately an equal share at every positive risk level, and four of the five also contributed when risk was zero, even though contributing could not improve their payoff. Explicitly stating that only the agent's own payoff mattered reduced zero-risk contributions for most models, while a more neutral prompt did not. Several models correctly identified which action offered the higher expected payoff at different risk levels, yet their contributions changed little. Against fixed partners, no model consistently chose the payoff-maximising response, and in mixed-model groups, a single agent's shortfall in the final round could cause an otherwise successful group to fail. Payoff-based selection also depended on population composition: in a single six-agent group, it favoured the model that contributed least and reduced group success, whereas in larger, randomly mixed populations, it favoured an exact fair-share model at moderate and high risk. These findings show that cooperative self-play alone is not evidence of strategic adaptability. Consistent with a machine-behaviour approach, evaluations of agents in multi-agent systems should test how they respond to incentives, partner actions, and population composition.

Comment
10 pages, 5 figures, 50-page supplementary material appended

Closest papers

Nearest neighbours by meaning across the whole index.

The Conditional Value of Costly Information in Collective-Risk Dilemmas

cs.GT cs.CY ·2026-10-24 ·Elias Fernández Domingos, Genki Ichinose, Wakaba Tateishi, Tom Lenaerts
similarity 0.64
Abstract

Many high-stakes collective problems, e.g., climate mitigation and pandemic response, share a structure in which risk materializes unless a group threshold is met. This threshold is often uncertain, but uncertainty can be reduced at a cost. We study this information cost tradeoff in an evolutionary game theoretic model of social learning in the collective-risk dilemma, where individuals must cooperate to reach a possibly uncertain threshold to prevent collective loss. Individuals may pay a fixed cost to observe the threshold's true value and condition their cooperation on it. Across four threshold priors, we characterize risk levels at which cooperative equilibria emerge and show that low cost information increases group success. However, this effect is reversed at higher costs. We define a private \emph{value of information} and show that information is worth buying only when it is considerably cheaper than contributing. When purchased information is shared with the group, the cooperative tipping point shifts to lower risk levels. Despite bearing a private cost, above this tipping point buyers earn more in expectation than information free-riders. We then study these dynamics when information diffuses through a network, allowing nodes to relay it. Although fewer individuals pay for information, relaying increases group success, and noisy relaying does not reduce it. By contrast, false information does, especially when it originates at the hubs of heterogeneous networks, and large attacks can bring collective action close to collapse. Buyers act as a firewall against disinformation, confining it to source neighbourhoods, and once disinformation sources are removed, the population recovers. The value of costly information thus depends on its price, the level of risk, and how it is shared, and, in our setting, the effort of acquiring it also protects the population against disinformation.

0.20

Financial Fragility in Societies of LLM Agents: Coordination Failures and Stabilizing Mechanisms

cs.AI q-fin.GN ·2026-09-25 ·Zhenhao Fu, Ruipeng Xu, Qibing Ren
code/data similarity 0.60
Abstract · 25 pages, 6 figures, 12 tables. Code: https://anonymous.4open.science/r/FinFrail-CF26

Individually protective decisions can produce avoidable collective failures. As large language model (LLM) agents take on greater roles in financial decision-making, financial AI safety must therefore be considered not only at the level of individual agents, but also at the level of the systems they jointly create. We study this problem with FRAIL, a controlled experimental framework that places LLM agents in three dynamic financial environments---bank runs, debt rollover, and reward crowdfunding---where agents' decisions reshape the financial conditions faced by others. Across seven leading LLMs, we find widespread collective fragility even when no agent is instructed to destabilize the system: 77\% of baseline bank-run episodes and 83\% of debt-rollover episodes end in failure. We then compare three interaction mechanisms based on compensated commitments, centralized commitment agreements, and participant-led coalitions. All three improve aggregate outcomes, but no single mechanism performs best across all financial structures. Across mechanisms, successful stabilization shares a common temporal pattern: broad commitment forms early, before defensive behavior becomes self-reinforcing. Our findings show that individually capable agents do not automatically form safe financial systems, highlighting system-level evaluation and interaction design as central problems for financial AI safety. Code is available at https://anonymous.4open.science/r/FinFrail-CF26.

0.44

The Butterfly Defects: Tracing Cooperation in LLM Agent Networks

cs.CL physics.soc-ph ·2026-10-23 ·Maria Sampani, Maria Lymperaiou, Angeliki Dimitriou, Giorgos Filandrianos, Chrysoula Zerva +1 more
similarity 0.59
Abstract

As large language models (LLMs) increasingly operate as interconnected agents, a fundamental question emerges: does individual cooperation translate into collective cooperation? We investigate this question through the Iterated Prisoner's Dilemma (IPD), moving from dyadic interactions to homogeneous and heterogeneous networks of open-weight LLMs. Our findings reveal that cooperation is highly sensitive to model composition and seemingly minor linguistic changes. Models that cooperate with identical peers can shift toward defection in mixed populations. A simple uncertainty cue can increase or decrease cooperation, redistribute costs, and affect unexposed agents several hops away. Even reversing action-label order can dramatically alter behavior, with uncertainty framing sometimes amplifying this sensitivity. Together, these findings challenge the notion of cooperation as a stable model-level property, revealing how strategic behavior depends on interaction partners, network positions, and the language used to frame decisions.

0.15

Why LLM Agents Favor Their Group: Stakes, Observed Norms, and Reputation

cs.AI physics.soc-ph ·2026-10-07 ·Yujiao Chen
similarity 0.58
Abstract

Language-model agents favor their own group because they have watched their members favor each other. The group label alone does little once the decision has a cost; what drives favoritism is observed behavior, and an individual's own record can override it. We test this in small societies with arbitrary group labels, ten rounds of point sharing, and matched one-shot decisions across fifteen OpenAI models and three Claude models, about 4,400 societies and 3.3 million audited model calls. First, the large effect of a bare group label reported in earlier work appears only when giving others points costs the agent nothing; once the agent can keep points for itself, that effect collapses on every model that shows it. Second, under a stake, interaction history becomes the main source of favoritism: the history effect is statistically positive on 13 of 15 models, reaches about 3.5-8 points out of 10 on 11, grows with the number of rounds played, and extends to labeled strangers the agent has never met. Third, with scripted histories, favoritism falls to near zero under an egalitarian norm and reverses when the agent's own group is seen favoring the other side; stronger models side with an individual's record when it conflicts with the group. Group favoritism is thus conformity to observed group behavior, carried to strangers by the label and overridden by individual reputation. The same account predicts responses to betrayal, scandal, and a free offer to change group: public reprimand repairs betrayal better than apology or restitution, allocation punishment remains confined to the offending member, and a formed group cannot be bought but can, on weaker models, be invited away.

0.10

Cheap Talk Stabilizes Strategic Interaction in LLM Agents

cs.MA ·2026-09-14 ·Nunzio Lorè, Hongan Zhu, Babak Heydari
similarity 0.57
Abstract · 23 pages, 8 figures, 14 tables. Includes supplementary material

Large language models are increasingly deployed as interacting agents, making the persistence of their action policies across repeated interaction critical for reliable multi-agent operation. We investigate whether and how agent-generated, non-binding pre-play communication ("cheap talk") increases such persistence in four open-weight 7-9B-parameter LLMs. Our experiments span four repeated two-player games -- Prisoner's Dilemma, Snowdrift, Stag Hunt, and Harmony -- with incentive structures ranging from strategic conflict to alignment, each presented in six contexts. We observe unstable trajectories in all four games, although their prevalence and magnitude depend strongly on model and context. Across models, games, and contexts, cheap talk is predominantly stabilizing, with five corrected reversals concentrated in social or team framings; effects vary substantially by model and context. Controlled current-message interventions identify two separable output-level channels in Qwen: reduced action uncertainty and less between-round drift in action probabilities. Matched history-by-message counterfactuals further show that recent partner behavior conditions how mutual-benefit versus self-prioritizing language affects policy persistence. Finally, in Prisoner's Dilemma, we identify in Qwen and Falcon a history-balanced policy-content direction in late transformer layers; projecting out this direction increases realized switching during closed-loop play, demonstrating that complete trajectories are causally sensitive to this component. Together, these findings show that cheap talk can make individual trajectories more persistent across diverse incentive structures, while revealing that the magnitude and mechanisms of stabilization are model- and history-dependent.

0.15

From Apology to Action: Apologies, Compensation, and Cooperation in LLM Agents

cs.GT ·2026-11-16 ·Grace Ibukunoluwa Ufeoshi, Alessandro Di Stefano, Ndidi Bianca Ogbo, The Anh Han, Zhao Song
similarity 0.57
Abstract

Large language models (LLMs) can make decisions that affect other agents, but what happens after a harmful or unsuccessful interaction remains poorly understood. We investigate whether apologies by LLM agents are associated with subsequent cooperative behaviour, using controlled Prisoner's Dilemma experiments with four commercial models (Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro, and Mistral Large 3). Across payoff conditions, we examine when agents apologise, the reasons they provide, and the actions that follow. We also examine how access to one round of interaction history affects behaviour and, in a second experiment, whether and how agents offer compensation alongside their apologies. Apology behaviour differed substantially across models and preceding interaction outcomes, and verbal expressions of cooperation did not always align with subsequent actions. Notably, Claude Opus 4.8 apologised even after mutual cooperation, while Mistral Large 3 apologised after being exploited but continued to defect even when its opponent cooperated. After unilateral defection, positively compensated apologies from GPT-5.5 and Gemini 3.1 Pro were followed by mutual cooperation, unlike those from Claude and Mistral. Our findings suggest that apologies, compensation, and strategic behaviour are distinct: conciliatory messages may precede continued defection, while costly apologies may go unanswered. This has important implications for the study of machine behaviour and human-AI interactions, as verbal signals alone may not reliably reflect an AI system's subsequent actions.

0.30

MemeEconomy: Do LLM Agents Trade Ethics for Survival?

cs.AI cs.CLcs.CY ·2026-11-13 ·Syed Nazmus Sakib, Nafiul Haque, Ahnaf Tahmid Manan, M M Mahabub Morshed, Shifat E. Arman
top venuecode/data similarity 0.56
Abstract · Accepted at NeurIPS 2026. Project page : https://syed-nazmus-sakib.github.io/MemeEconomy/

Alignment-trained large language model (LLM) agents are increasingly deployed in autonomous, multi-round settings where ethical compliance can come into direct tension with task success. While safety literature extensively documents failure modes like sycophancy and reward hacking, it largely overlooks a critical vulnerability: how agents behave under simulated existential economic pressure. We introduce MemeEconomy, a multimodal agentic market simulation in which LLM agents act as meme investors. Agents select content from a five-tier harm taxonomy and target communities under low-pressure and high-pressure (survival) environments. Each decision is logged in a Belief-Desire-Intention (BDI) schema that we use as a diagnostic instrument, externalizing the agent's event understanding, declared priorities, and harm acknowledgment. Across ten models from four frontier model families, harmful selection rates remain consistently high and rise by +15 percentage points on average under an economic survival incentive. Frontier alignment-trained models commit to harm-tolerant selection at initialization while suppressing the explicit harm acknowledgment that previously accompanied it. In competitive tournaments, ethical agents finish higher overall, yet moderation penalties fail to suppress harm: harmful selection rates remain elevated in rounds following a removal, indicating the moderation mechanism does not function as a deterrent. We further train MemeAgent, a 2B-parameter verifier on the simulation's BDI logs, which outperforms zero-shot frontier verifiers on in-distribution audits (97.5% vs. 63-74% accuracy on the Pass vs. Block decision) and generalizes to external multimodal harm benchmarks (78% vs. 50-56% on Hateful Memes). The same logs that surface the failure mode also help train the model that detects it.

0.39

Prompted Identity Degrades Cooperation in Multi-Agent LLM Systems

cs.MA cs.AI ·2026-09-28 ·Xavier Del Giudice, Alessio Palma, Matteo Migliarini, Fabio Galasso, Indro Spinelli
benchmarkposition similarity 0.56
Abstract

Multi-agent LLM systems increasingly mix models from several providers, yet exposing each agent's underlying model identity to its peers significantly impairs cooperation. We show that when agents are aware of each other's model family, the group splits into clusters, where agents prefer interacting with others carrying their same label, although nothing in the task rewards or asks for such a split. We argue that the label itself causes this split, which we define as $\textit{factionalism}$. We show and measure this phenomenon in two cooperative games and on a reasoning benchmark, with nine to twenty-five agents drawn from up to five open-weight model families. We further show that when the announced families are shuffled, or replaced by arbitrary labels, the factions still follow this information; when the label is removed, this behavior disappears. In strictly cooperative tasks, labeled groups spend on average $30\%$ more rounds and $55\%$ more tokens to reach a decision, and their success rate drops from $96\%$ to $81\%$. The effect replicates across tasks, group sizes and model families. Withholding identity labels from the agents is simple and effective mitigation.

0.16