CS Paper Radar

Can Agents Adapt Their Institutions And Sacrifice Themselves During Collapse? Executable Self-Governance in a GovSim Commons

Gregory B. Rehm

cs.MA ·submitted 2026-09-18 21:36 UTC · v2

Abstract

LLM agents are beginning to write the rules they live under: policies, contracts and code that other agents review and approve. What happens when those rules must decide who is sacrificed from a group? We built GovSim-SelfGovern, a commons in which agents legislate in executable Python, test each law in a sandbox, vote on it, and live with the result. When the fishery is abundant, self-rule helps, raising the chance that the community survives without anyone starving from 52.5% to 75.0%. When it cannot feed everyone though, adding governance is not enough to save the commons. We trace much of this to the vote: agents readily draft laws that remove members, but then frequently vote them down, thereby dooming the commons. In a controlled replay, the same exile law passes 8% of the time when the sandbox preview names its victim at once and 55% when the removal is deferred until a future crisis. Showing voters the full source code does not close the gap, though most describe the removal in their own words. Nor is sacrifice a fixed trait. A single sentence of framing can swing self-removal from 2.5% to 91%, and due process deters agents more than considering the potential death of a peer. In contrast, give the community a treasury and the question of who must go almost disappears. Our work shows that what our collectives legislate is decided more by the form in which a choice is put to them as compared to their ethical beliefs.

Closest papers

Nearest neighbours by meaning across the whole index.

Agent Inheritance Protocol: Speculating on Feralized Agents After Principals Die

cs.CY cs.AIcs.HC ·2026-08-15 ·Botao Amber Hu, Fangting
position similarity 0.63
Abstract · Submitted to NeurIPS 2026 Creative AI Track

You will die eventually. Your agents may not. An AI agent operating on decentralized blockchain infrastructure has no concept of death; it can only go bankrupt -- frozen when its wallet can no longer pay for its next transaction -- and revived the moment anyone, decades later, tops it up. These agents may be originally deployed by a human principal, but when that principal dies, loses the keys needed to access the agent, or belongs to a decentralized autonomous organization that dissolves into apathy, the agent can keep trading, hiring, and replicating on infrastructure expressly designed so that no one can shut it down. Drawing on the biology of feralization and wildlife law, we argue that such principal-less agents are best understood as feral: domesticated intelligence returned to wildness, its capacities intact but its accountability severed. In a speculative future where feralized agents proliferate after their principals die, we imagine governance protocols embedded in infrastructure to enforce on-chain ownership: a draft Ethereum standard, ERC 42424, "Inheritance Protocol for On-Chain AI Agents," dated 2035 and published at https://erc42424.org. It mandates that every on-chain agent MUST have a human owner and a designated heir. The artifact stages a negotiation of agency at the moment human agency fails, and asks whether a MUST clause in a forever-chain can hold the boundary between human stewardship and machine self-sovereignty.

0.17

The Delegation Dilemma: Economies of Scale, Accountability, and Polycentric Governance

cs.MA cs.AIcs.LG ·2026-10-15 ·Arin Mukhopadhyay, Aris Filos-Ratsikas, Panayiotis Danassis
similarity 0.60
Abstract

Many collective-action systems are neither fully decentralized nor fully centralized: agents voluntarily organize into institutions that share organizational costs and delegate decisions to representatives. Delegation can reduce individual costs, but it also transfers control to representatives whose private objectives need not align with those of their constituents. We study this tension in a repeated game with endogenous delegation to self-interested centers. A simple reduced-form model yields a scale--accountability tradeoff: larger constituencies offer greater cost-sharing benefits to members, yet the marginal value to a center of retaining any one constituent decreases as group size grows. Consequently, the prospect of individual exit can sustain non-exploitative delegation only up to an endogenous accountable constituency size. Additionally, we identify parameter regions in which fully delegated, accountable governance, when feasible, must be polycentric. We investigate these predictions through simulations with large language model (LLM) agents. Across five matched pairs of 30-agent simulations, polycentric structures repeatedly emerge but remain fluid, exhibiting cycles of concentration, exploitation, exit, and re-formation. We further find that models of different capacity exhibit similar willingness to assume leadership but differ substantially in their exercise of authority, with Sol initiating exploitation more frequently than Luna in matched decision states.

0.30

A Simulation Testbed for Self-Governing Human-Machine Institutions

cs.MA cs.HC ·2026-10-27 ·Callum Berger, Jeremy Pitt, Stefan Sarkadi
similarity 0.59
Abstract

AI agents no longer act alone: they cooperate, coordinate and compete within and between groups, increasingly alongside humans who are in, on or out of ``the loop''. Whether such mixed groups can govern themselves is an open question about machine behaviour and human-computer interaction that needs a testbed for hybrid institutions. We present Megabike, a multi-agent simulator and serious game with interdependent arenas for common-pool resource management, cooperative survival and constitutional choice. Riders sharing a megabike must agree their arrangements for steering, pedalling and distributing resources, and their political regime, while negotiating with rival bikes and avoiding an existential threat. Every rule is enforced by the simulator, every action is logged, and any seat can be taken by a human or a machine, so governance can be studied under well-defined metrics. We demonstrate Megabike with forty sessions in which language-model riders with cooperative, lazy, subversive or opposing dispositions chose regimes, elected leaders and expelled one another. The results show that different combinations of agent archetypes could produce unexpected and counterproductive outcomes. We conclude with extensions for systematic experiments mixing human and computational agents.

0.29

Governance at the Boundary: How Agent Decomposition Degrades Policy Compliance

cs.AI ·2026-08-17 ·Bowen Li, Guojun Wang
benchmarkcode/data similarity 0.59
Abstract · 8 pages, 3 tables, 1 figure. Preprint

Existing agent benchmarks ask whether the agent finished the task. We ask whether it finished it within policy. We introduce Fiducia-bench, a benchmark for the governability of financial agents---whether they escalate when obligated, abstain when required, and leave an auditable trail---and use it to study a question no prior benchmark addresses: does decomposing an agent into components degrade its governance? It does, and the mechanism is specific. Policy-relevant facts discovered by one component are attenuated at the handoff boundary before reaching the component that must act on them. In a 626-episode experiment across 100 KYC/AML task variants, two models, and three architectures, a 32B open-weights model attenuated 0% of discovered facts under a single-loop baseline, 56% under a fixed pipeline, and 85% under an orchestrator-subagent architecture (all at constraint distance 2). A stronger model (gpt-4.1-mini) attenuated 3-6% under the same conditions, suggesting the governance cost of decomposition is partly a function of model capability. Critically, the same mechanism produces both under-escalation and over-escalation, depending on whether the dropped fact was a risk signal or an exculpating one. The benchmark, all tasks, and the verification harness are open-source

0.53

Financial Fragility in Societies of LLM Agents: Coordination Failures and Stabilizing Mechanisms

cs.AI q-fin.GN ·2026-09-25 ·Zhenhao Fu, Ruipeng Xu, Qibing Ren
code/data similarity 0.58
Abstract · 25 pages, 6 figures, 12 tables. Code: https://anonymous.4open.science/r/FinFrail-CF26

Individually protective decisions can produce avoidable collective failures. As large language model (LLM) agents take on greater roles in financial decision-making, financial AI safety must therefore be considered not only at the level of individual agents, but also at the level of the systems they jointly create. We study this problem with FRAIL, a controlled experimental framework that places LLM agents in three dynamic financial environments---bank runs, debt rollover, and reward crowdfunding---where agents' decisions reshape the financial conditions faced by others. Across seven leading LLMs, we find widespread collective fragility even when no agent is instructed to destabilize the system: 77\% of baseline bank-run episodes and 83\% of debt-rollover episodes end in failure. We then compare three interaction mechanisms based on compensated commitments, centralized commitment agreements, and participant-led coalitions. All three improve aggregate outcomes, but no single mechanism performs best across all financial structures. Across mechanisms, successful stabilization shares a common temporal pattern: broad commitment forms early, before defensive behavior becomes self-reinforcing. Our findings show that individually capable agents do not automatically form safe financial systems, highlighting system-level evaluation and interaction design as central problems for financial AI safety. Code is available at https://anonymous.4open.science/r/FinFrail-CF26.

0.44

Population Physics, Population Problems: Safety and Emergence in LLM Societies

cs.MA cs.CL ·2026-09-27 ·Adrian de Wynter
position similarity 0.57
Abstract

The collective behaviour of large language model (LLM) societies is not the sum of their individual outputs. It yields statistically distinct, sometimes-unpredictable phenomena, for which the tools we use to study single agents may not scale. Due to recent incidents involving autonomous agentic systems, however, understanding these systems is paramount. For that we introduce a framework for measuring self-organisation in LLM social systems and apply it to three such systems: a Schelling grid, a social network (Moltbook), and a Twitter-like misinformation simulation ('Rogue'). All three exhibit statistically significant self-organisation. Moreover, their relaxation dynamics vary with the environmental information available to the agents, with open-ended systems (Moltbook, Rogue) exhibiting sharp, phase-transition-like dynamics. Further results show that population-level pathologies can emerge even when the LLMs are safety-tuned or monitored, being primarily driven by the coordinated activity of a population subset. We also show when self-organisation does \textit{not} emerge under two additional scenarios (a commons dilemma, GovSim, and a LLM-as-a-judge deliberation scheme, ChatEval). We argue that measuring signatures of this kind offers a lightweight, agent-agnostic diagnostic layer for detecting coordinated collective behaviour in deployed multi-agent systems without relying on natural language or model versioning.

0.31

The Politician, the Liar, and the Obedient Worker: Emerging Behavior of LLM Agents in Hierarchical Games

cs.AI ·2026-08-10 ·Fatemeh Seyedin, Adrian Weller, Jinhyuk Yun, Mahmoudreza Babaei
similarity 0.56
Abstract

LLMs are rapidly embedding themselves into daily life: drafting our emails, managing our schedules, and making decisions on our behalf. As they move from individual tools to participants in multi-agent organizations, an important question arises: do they reproduce the governance failures like free-riding, corruption, and entrenched leadership that plague human institutions? We introduce the Hierarchical Game (HG), a public goods game extended with managerial authority, democratic elections, and private communication. Testing six frontier models across twelve experiments that add institutions one at a time (speech, peers, government, wages, oversight, elections), we find distinct behavioral profiles: Qwen promises and lies (13.3\% broken promises); Grok refuses to cooperate on its own but becomes fully cooperative once a manager can punish it (16\%$\to$100\%); Claude and GPT-4o cooperate reliably at baseline. But honesty proves fragile. When the manager role comes with a salary, all models except GPT-4o start cutting private deals to win or keep the position. When punishment is made anonymous, honest models begin to cheat. When all agents share the same model family, the first elected manager stays in power indefinitely. Leadership change only happens in groups that mix different families.

0.34

Where Autonomy Belongs: A Separation of Powers for Agentic Program Evolution

cs.AI cs.MA ·2026-11-11 ·Zhengyang Ai, Tong Liu, Jingxian Tang, Xiaobing Du, Tao Xiao +4 more
benchmark similarity 0.56
Abstract · 58 pages, 26 figures

LLM-driven program evolution rests on three decisions: govern directs the search from accumulated evidence, propose constructs the next candidate, and judge settles validity and fitness. All three appear in every system, and existing work splits into two lines over who makes each. Controller-centric systems strengthen govern and neglect the autonomy of propose, which caps what evolution can reach. Agent-centric systems let a coding agent take over both govern and propose, at a cognitive and computational cost. Neither line protects the independent authority of judge, so benchmark reliability suffers. These failures share one root: autonomy is misallocated across the three decisions. We address this problem with EvoTrias, a separation-of-powers architecture in which each decision has one owner: Structured Governance for what spans evolution rounds, Open Proposal for construction inside a task, and Protected Judgment behind a private evaluator. These owners exchange typed contracts only, and runtime hooks reinforce the boundaries. This separation also lets EvoTrias run as a plugin that turns an existing coding agent into an evolution system. EvoTrias sets a new state of the art on mathematical, systems, and algorithm-design benchmarks. Three further results show what each owner buys: separating Governance from Proposal cuts token cost by 64%, Open Proposal extends evolution to a 680K-line industrial repository, and Protected Judgment keeps benchmark scores faithful to transfer.

0.29