CS Paper Radar

Emergent aggregation from collective foraging

Gorka Muñoz-Gil, Andrea López-Incera, Vide Ramsten, Giovanni Volpe, Thomas Müller, Hans J. Briegel

cs.LG cond-mat.stat-mechcs.MAnlin.AOphysics.bio-ph ·submitted 2026-08-28 08:08 UTC · v1

Abstract

Collective behaviour in living systems is usually modelled as the outcome of a \emph{direct} social drive: agents are rewarded, or hard-wired, to align with or approach their neighbours. Here we show that aggregation can instead emerge from an \emph{indirect} objective. We let reinforcement learning foragers, initially performing a random walk, optimize their dynamics from a purely individual reward for finding replenishable targets, while perceiving only their conspecifics and never the targets themselves. As the visual range grows, the agents undergo a sharp crossover from an environment-tuned individual search to a scale-agnostic collective one, and this crossover coincides with the onset of spatial aggregation. Thus a collective phase arises as a by-product of optimal foraging, without any direct reward for grouping. A minimal analytical first-passage model reproduces the transition as a crossover between the two search strategies. Our results identify indirect, resource-driven reward as a generic route to emergent collective phenomena.

Comment
11 pages, 8 figures

Closest papers

Nearest neighbours by meaning across the whole index.

A self-organizing mechanism for collective foraging: recruitment cascades couple Levy search to a clustered maximum-entropy resource field

cs.DC physics.soc-phcs.SImath-ph+1 ·2026-10-17 ·Galen J. Wilkerson
code/data similarity 0.70
Abstract · 21 pages, 19 figures (7 main + 12 SI), 1 tables, Code and data: https://github.com/galenwilkerson/collective-foraging-cascades

Foraging recruitment is modeled as a contagion: an agent that finds a resource triggers a linear-threshold cascade on the group's communication network. Because activation is degree-normalized, crowding shuts the cascade off, so the swarm aggregates on a resource and then disperses to explore again. Acting on the cascade moves agents in space, and that motion rewires the network, so recruitment and exploration are one coupled process; removing this coupling alone abolishes the self-limitation. Recruitment spreads only within a connectivity window, bounded below by percolation onset and above by degree-blocking. The swarm self-organizes to the near-critical edge of this window -- self-organized quasi-criticality: recruitment sits near but below the critical point without settling onto a clean, size-invariant critical state, and the activity spectrum has a finite correlation time (Lorentzian-like, not 1/f), consistent with local rather than global criticality. Recruitment raises harvest up to 2.7x over solitary search on sparse, clustered resource fields and loses (down to 0.37x) on abundant or uniform ones; because resources generically aggregate, the favourable regime is the common one. The empirically motivated Levy search and the resource field it exploits are both least-committal: the search a maximum-caliber walk, and the field a maximum-entropy point process whose generic, aggregation-set sign is clustered (attractive). The search exponent that maximizes harvest tracks the field's fractal dimension (a heavier-tailed, more ballistic search for more strongly clustered fields). These are one coupled mechanism: heavy-tailed Levy exploration crosses the field's sparse voids while recruitment cascades exploit its clusters, so one maximum-entropy geometry binds exploration and recruitment into a single self-organizing cycle.

0.40

Competitive exploration in structured environments

cs.GT physics.soc-phq-bio.PE ·2026-10-26 ·James P. Wolfe, Joshua B. Plotkin
code/data similarity 0.69
Abstract · 41 pages, 15 figures, 5 tables (19 pages, 6 figures, 3 tables main text; 22 pages, 9 figures, 2 tables SI Appendix). Code: https://github.co…

Organisms must constantly weigh exploiting the resources they know against exploring for better ones. When foragers compete, the best behavior for each forager also depends on what its competitors do. How does competition change the strategic balance between exploration and exploitation? Here we address this question in a minimal model: two $ε$-greedy temporal-difference learners on a one-dimensional lattice, who start at a small reward at one end and are unaware of a larger reward at the other. For a lone forager, the optimal exploration rate increases monotonically with the value of the distant reward, and with its distance up to a critical distance. When two foragers share the lattice and split rewards, the choice of exploration rate becomes a game whose Nash equilibria have both foragers exploring, both staying, or one exploring while the other stays (anti-coordination). Under local best-response dynamics, pairs of foragers often converge on both exploring or both staying, even when anti-coordination would be socially optimal. This social dilemma arises for a wide range of parameters, and it persists even in a simplified model without explicit spatial structure. A minimal, naturalistic signal, in which one forager informs the other that it has found the distant reward, can remove many of the dilemmas in which both foragers explore, and can produce a Nash equilibrium whose collective payoff exceeds that of every outcome available without signaling.

0.34

Spatial Pattern Formation from Multi-Agent Learning in Public Goods Dilemmas

cs.MA cs.GTcs.LG ·2026-10-08 ·Yefei Zhang, Yuxuan Zhao
similarity 0.63
Abstract

Spatial public goods models show that prescribed movement toward richer locations can generate spatial patterns. We ask how such patterns emerge when agents learn where to move and how learning rates shape their consequences for collective welfare. Fixed populations of cooperators and defectors independently learn movement policies using tabular Q-learning and local observations. Cooperator learning generates clusters around resource peaks, while co-adaptation changes their strength and motion. At a fixed training budget, the largest welfare losses occur when cooperators learn at high rates and defectors at low rates. In part of this regime, learned policies also generate traveling bands supported by a shared directional preference. The conditions supporting travel change with further training, so these patterns reflect training history rather than an established asymptotic outcome. Across the tested learning-rate conditions with cooperator learning, mean collective welfare falls below random movement because increased crowding outweighs gains in resource benefit. Charging agents for the crowding they impose on others during learning recovers much of the welfare loss in the tested conditions. These results connect learning rates to the emergence and welfare costs of spatial organization driven by individual rewards.

0.36

A learner-free swarm approximates the explore-exploit equilibrium allocation: A self-organized attractor in place of reinforcement-learning machinery

cs.LG cs.AIcs.SImath-ph+2 ·2026-10-18 ·Galen J. Wilkerson
code/data similarity 0.63
Abstract · 26 pages, 13 figures, 5 tables, code and data: https://github.com/galenwilkerson/self-organizing-reinforcement-learning

Reinforcement learning (RL) solves the explore-exploit problem by training a learner on reward; its multi-agent, resource-sharing form has an equilibrium allocation as its optimum -- the ideal-free distribution (IFD), the Nash equilibrium of a congestion game. We show a minimal swarm approximates it with no learner at all. We present a decentralized, learner-free method for this problem: agents explore by heavy-tailed (Levy) flights, recruit neighbours to a discovery through a degree-normalized threshold cascade, and disperse as a patch crowds. The method drives the collective to the shape of the equilibrium allocation (pooled spatial matching slope 1.14, R^2=0.61) with no training, no stored parameters, and no reward in its dynamics; it reaches this as a self-organized attractor and holds it, re-finding it when the resources move -- though it reaches the shape, not the exact equilibrium, and stays measurably exploitable (Nash-gap 1.46). Coupling recruitment to movement causes the allocation: removing it collapses the matching (slope 0.006). Against Proximal Policy Optimization (PPO) learners (feedforward, centralized-critic, graph message-passing, and recurrent) trained in the same sensing-limited environment, the method at zero training is the only arm left unbiased w.r.t. the IFD gradient (slope ~1, CI containing 1, vs learners 0.69-0.97, all under-concentrating) and ties the best learner's harvest where resource is clustered, while the learner collects more where resource is scarce (0.33 vs 0.22) and holds a tighter per-capita intake. Learning buys precision and transfer across reward structures, not a better allocation shape. A companion paper develops the mechanism.

0.42

Exploration via Exploitation: The Blessing of Reward Diversity in Personalized Federated RL

cs.LG ·2026-11-07 ·Jiin Woo, Gauri Joshi, Yuejie Chi
theory similarity 0.61
Abstract

While federated reinforcement learning (RL) has traditionally guaranteed high efficiency in homogeneous environments, collaboration is often hindered when agents possess distinct goals. In this paper, we flip this paradigm, framing reward heterogeneity not as a hurdle to consensus, but as a structural asset that catalyzes exploration. We first propose Personalized Federated Upper Confidence Bound Value Iteration (PF-UCBVI), which achieves optimal linear speedup by decoupling shared dynamics from personalized objectives. To eliminate the risks of explicit exploration, we then introduce Personalized Federated Exploration-Free Value Iteration (PF-EFVI), a purely greedy algorithm that leverages reward diversity to ensure state-action coverage. We prove that PF-EFVI attains logarithmic regret without explicit exploration bonuses under sufficient reward diversity. Our results reveal that agent disagreement shifts the exploration burden from temporal complexity to spatial diversity, enabling efficient collective learning without explicit exploration.

0.05

Self-Referenced Social Preferences: Cooperation without Observing Others Rewards

cs.AI cs.MA ·2026-10-06 ·Mohamed Ayman Mohamed, Harshil Kotamreddy, Marcos Menon Jose
similarity 0.61
Abstract · Under Review

Social preferences can promote cooperation in multi-agent reinforcement learning, but existing approaches often require agents to observe the rewards of their peers. In many real-world interactions, however, an agent can, as humans do, observe others' behavior and outcomes without access to their private reward signals. We introduce self-referenced social preferences, in which each agent learns a model of its own reward, applies it to other agents' observed transitions to assess their outcomes from its own perspective, and feeds these self-referenced assessments into standard social preferences. We study two ways to incorporate these assessments: modifying the learning reward, or using them to weight policy updates. We evaluate the approach on three sequential social dilemmas, Escape Room, Clean Up, and Commons Harvest, which require volunteering, public-good contribution, and resource restraint, respectively. Across all three environments, agents learn cooperative behavior without observing others' rewards, including in settings where independent learners fail to cooperate, and frequently achieve more equitable divisions of jointly produced returns than agents with access to true rewards. The effective integration point depends on the social preference: inequity aversion works best in the reward together with a value look-ahead, whereas a purely benevolent preference benefits from policy-update weighting. Under partial observability, the policy-update approach continues to support cooperation. These results show that explicit access to other agents' reward signals is not necessary for learning cooperative behavior: social preferences can instead be grounded in self-referenced assessments of others' outcomes derived from their observed behavior.

0.31

Emergence, Asymmetry, and Value in AI Networks: An Anoptical Perspective

cs.CY physics.soc-phcs.MA ·2026-10-01 ·Olivier Auber
published similarity 0.61
Abstract · To appear in T. Veloz and C. Rittberg (Eds.), AI and Human Values, Springer

This chapter presents exploratory experiments suggesting that asymmetry and functional value hierarchy emerge among LLM-based agents sharing the same context, although they are equivalent and interact within a symmetric network. We use the Poietic Generator, a "no-goal" multi-agent environment with a shared real-time space, a symmetric topology, and no imposed objectives. First tested with humans, it has hosted since 2024 multimodal LLM agents that observe the global image they co-construct and act locally. We ran four phases. Unconstrained, agents spend tokens freely to invent forms that distinguish them individually rather than serve collective emergence. We then introduced bounded rationality, Dessalles's Simplicity Theory (maximizing unexpectedness), and Friston's Free Energy Principle (minimizing prediction error), and observed collective coherence emerge. Early signs of attentional stratification appear under bounded rationality alone; once prediction accuracy is made mutually visible, this proto-hierarchy becomes legible and self-reinforcing. The unconstrained phase challenges the allotelic view of LLMs as purely user-directed instruments: left unconstrained, the agent maximizes the computation from which its provider profits. The later phases suggest that attentional asymmetry and value hierarchy are an irreducible emergent property of interactive predictive systems even under symmetric conditions. If so, aligning AI with human values cannot rely on individual-agent engineering alone. Drawing on the idea that networks are structured by invisible "anoptical" perspectives, akin to the vanishing point of Renaissance perspective, we propose minimal legitimacy criteria that make emergent asymmetries visible and contestable. Cultivating judgment, human and perhaps artificial, to perceive and contest these structures matters as much as engineering the agents themselves.

0.38

Gossip Agents: Reputation through Opinion Dynamics over Evolving Networks

cs.MA ·2026-10-15 ·Yutong Wu, Arlei Silva
similarity 0.60
Abstract · 10 pages, 8 figures

Sustained cooperation among self-interested learning agents is essential in many shared environments. Reputation systems are among the best-studied mechanisms for promoting such cooperation. Existing multi-agent reinforcement learning approaches, however, typically assume that reputation is either globally available to all agents or derived solely from an agent's own observations, leaving largely unexplored how reputational information propagates between agents. We study gossip, the exchange of information about third parties, as a mechanism for reputation diffusion over dynamic communication networks. Agents form private opinions from limited observations and exchange them with their neighbors following DeGroot opinion dynamics, while the communication network changes as agents interact and move. A tunable broadcast radius controls the reach of information, spanning private reputation, local gossip, and globally shared reputation. We study this framework in two \(n\)-player social dilemmas with distinct network dynamics: a Prisoner's Dilemma with randomly resampled interaction groups and Harvest, where communication neighborhoods emerge from agents' movements. Our results show that the reach of reputational information shapes cooperation in both environments. Moreover, in Harvest, reputation diffusion increases environmental rewards and leads to more peaceful and sustainable behavior.

0.28