CS Paper Radar

Feedback autocratic strategies in stochastic games

Masahiko Ueda

cs.MA physics.soc-pheess.SY ·submitted 2026-10-27 05:52 UTC · v1

Abstract

A stochastic game is an optimal control problem for multiple agents in a stochastic environment. Stochastic games contain important practical problems, related to environmental issues or traffic flows. If the number of environmental states is one, such stochastic games are called repeated games. In repeated games, several strategies which unilaterally control payoffs have been proposed, and these strategies work in a mechanism similar to feedback control. In this paper, we generalize these strategies as feedback strategies in stochastic games, and investigate their properties. In particular, we provide a necessary and sufficient condition for such feedback strategies to work as controllers of payoffs under some assumptions. We demonstrate the performance of such feedback strategies in several examples related to social dilemmas. Our findings connect the theory of payoff control in repeated games to the theory of feedback control in stochastic games.

Comment
19 pages, 2 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Performance of Zero-determinant Strategies in Repeated Games without Discounting

cs.MA physics.soc-pheess.SY ·2026-10-02 ·Masahiko Ueda
similarity 0.67
Abstract · 4 pages, 1 figure

Zero-determinant (ZD) strategies are a class of strategies in repeated games, which unilaterally control payoffs. It has been shown that several ZD strategies promote cooperation in social dilemma games. It has been widely believed that the performance of ZD strategies is expressed as ``unilaterally enforce linear relations between payoffs''. However, in order to interpret properties of ZD strategies in repeated games without discounting, previous studies assumed that the limit of the Cesaro average of the probability distribution of the action profile exists. Here, we explain the performance of ZD strategies without this assumption, which leads to a stronger result than previous ones.

0.41

Social Laws for Multi-agent Coordination in Stochastic Environments

cs.MA cs.AIcs.LG ·2026-09-16 ·Rolando Fernandez, Caleb Probine, Tyler Lee, Jeffrey Chen, Erez Karpas +3 more
published similarity 0.63
Abstract · Appeared at the RIPL Workshop as part of ICAPS 2026

In multi-agent environments, coordinating agents to prevent interference and ensure robust individual performance is a critical challenge. Previous research on social laws for multi-agent systems has primarily focused on deterministic, goal-based settings. This paper extends the concept of social laws to stochastic, reward-based environments, proposing a formalism for defining and verifying their robustness under various conditions. We introduce the notion of $α$-robustness, a measure of the guaranteed utility each agent retains while pursuing its optimal single agent policy, assuming all agents obey the social law. We then present an approach for robustness verification of social laws in stochastic settings, based on a reduction to solving a series of Markov decision processes. Empirical evaluations on toy environments illustrate the potential of our framework.

0.53

Receding Horizon Control and Dissipativity - Optimal Control, Games and Uncertainty

eess.SY math.OC ·2026-09-29 ·Sophie Hall, Jonas Schießl, Sergio Grammatico, Timm Faulwasser
publishedsurvey similarity 0.60
Abstract · To appear at IEEE CDC 2026

We provide a tutorial overview of receding horizon control across deterministic, stochastic, multi-agent, and game-theoretic settings. We contrast the minimizer of an optimal control problem with the equilibrium of a game in terms of cost and constraint handling to motivate the difference of MPC schemes involving multiple agents/players. Interestingly, dissipativity theory and the turnpike property have turned out to be a unifying thread and system-theoretic backbone across all these schemes. This paper is the first to give an overview of results, summarizing fundamental insights, drawing parallels, and highlighting technical differences in dissipativity and turnpike analysis across economic, game theoretic, and stochastic settings. We also explore recent trends in numerical methods for games. We conclude by pointing to a list of open challenges.

0.32

Refundable Deposits: How to Restore Cooperation in Finitely Repeated Games

cs.GT econ.TH ·2026-08-27 ·Giulio Salizzoni, Domenico Mergoni Cecchelli, Edward Plumb, Maryam Kamgarpour, Galit Ashkenazi-Golan
similarity 0.60
Abstract

While infinitely repeated games admit a rich set of Nash equilibria, finitely repeated games typically have a much smaller and often inefficient one. We show how to enlarge this set using deposits: in each period a player may place a refundable sum with a neutral intermediary, returned when the game ends and forfeited following a deviation. Paying these deposits is voluntary and incentive compatible at every stage, so no commitment by the players is assumed, the only commitment required being that of the intermediary to a refund rule fixed before play begins. The mechanism sustains payoff profiles more efficient than those of the standard equilibria, without altering the underlying game and without transfers between players. We demonstrate it on the prisoner's dilemma, a congestion game, and a public goods game, all settings where cooperation cannot emerge in the standard finitely repeated version. We also apply it to a dynamic common-pool resource, suggesting that the construction extends beyond repeated stage-games.

0.18

How Learning Selects Equilibria and When It Leads to Collusion

cs.GT ·2026-10-26 ·Hanzhang Wang, Manxi Wu
benchmark similarity 0.60
Abstract

We study how learning algorithm design shapes equilibrium selection in infinite-horizon discounted Markov games with finite states and actions. Each episode, players choose smooth best responses to beliefs about opponents' policies, test them using noisy observations, and resample rejected beliefs or explore at rates determined by increasing transformations of their discounted values. We show that stochastically stable states induce approximate stationary Nash equilibria maximizing the minimum transformed discounted value across players, with transformations governing which players selection favors. In repeated games with bounded recall of noisy public signals, longer recall weakly increases the selected minimum transformed value. In repeated Cournot games, every stochastically stable state induces an approximate public perfect equilibrium whose minimum discounted profit across firms exceeds the Cournot-Nash benchmark. Collusion is therefore inevitable under these conditions and learning dynamics. For symmetric firms, selected profits can approach the monopoly benchmark arbitrarily closely. These results offer a novel equilibrium-selection perspective on algorithmic collusion.

0.17

Cycle structure of best-response dynamics in random bimatrix games

cs.GT ·2026-10-21 ·Georgios Nikolaos Karelas, Gábor Lugosi, Rui-Ray Zhang
similarity 0.59
Abstract

We study the best-response dynamics in games with random payoffs. We consider two-player bimatrix games, with $n$ strategies each. We assume that payoffs come from a continuous distribution, and study the best-response dynamics of these games, after imposing some structure on their payoff matrices. In particular, we focus on symmetric games, antisymmetric games, common-interest games, zero-sum games, and games with independent payoffs. When two players take turns in playing the best response to the opponent's action, they eventually end up cycling through the same sequence of actions. The main focus of the paper is the cycle structure of large random games. In particular, we study the number of cycles, the number of cycles of a certain length, the length of the longest cycle, and related quantities. We show that the structure of the game has an important impact on the typical cycle structure. Our arguments are based on a representation of the best-response dynamics of these games as directed bipartite graphs, and on the cycle structure that arises in them.

0.10

Entropic Risk-Sensitive Evolutionary Learning and Equilibrium Selection in Coordination Games

eess.SY cs.GTcs.MA ·2026-09-08 ·Solaleh Mohammadi, Xiang Gao, Kaiqing Zhang
publishedbenchmark similarity 0.57
Abstract · Preliminary version accepted to IEEE CDC 2026

We study risk-sensitive evolutionary learning dynamics and their long-run equilibrium selection behaviors in coordination games. Agents' risk attitudes enter through the classical entropic risk measure, which evaluates opponent-induced payoff uncertainty and feeds into noisy best responses under two standard revision protocols: best response with mutations and logit choice. We first analyze $2\times 2$ coordination games in both single-population symmetric and two-population asymmetric settings. In the single-population setting, unlike the risk-neutral case where the dynamics are known to favor the risk-dominant equilibrium, we show that risk sensitivity can change the stochastically stable outcome: a greater risk-seeking attitude favors the payoff-dominant equilibrium, while a greater risk-averse attitude favors the maximin equilibrium. Thus, the population's risk attitude may act as a control knob for long-run equilibrium selection. In both population settings, we also identify a robust regime: any super-dominant equilibrium is stochastically stable for all risk attitudes, under both protocols, and across populations. We further extend the single-population analysis to symmetric $k$-action games, which include symmetric $k$-action coordination games as a special case, under risk-sensitive best response with mutations. In this setting, we show that, for sufficiently large populations, sufficiently risk-seeking agents uniquely select the strongly payoff-dominant equilibrium when it exists, whereas sufficiently risk-averse agents uniquely select the strongly maximin equilibrium when it exists. These results show that entropic risk sensitivity may serve as a systematic mechanism for steering equilibrium selection in evolutionary games, beyond the classical risk-neutral benchmark.

0.33

Bayesian Replicator Dynamics and the Inversion of Strict Dominance in Bimatrix Games

cs.GT cs.MA ·2026-10-26 ·Jaime Ruiz-Serra, Nihat Ay, Michael Harré
similarity 0.57
Abstract · 18 pages, 3 figures

Replicator dynamics, the central model of evolutionary game theory, describe populations evolving under selection and agents learning by trial and error. They also admit a third use: continuous-time Bayesian inference, in which an agent planning against an opponent accumulates evidence for each candidate strategy. Here we show that this use divides all two-player games into two classes. With the standard expected-utility payoff, the dynamics can be expressed as Bayesian updating only in games (with any number of actions) where no player has a dominant strategy. The logarithmic payoff of Ay and Erb (2005) admits the Bayesian representation in every game. Coordination, anti-coordination, and cyclic games share qualitative dynamics under both payoff definitions, with equilibria displaced and selection thresholds shifted. In contrast, the flows of every game with a dominant strategy are reversed between the payoff definitions, with the strictly dominant strategy eliminated under logarithmic payoffs. Cooperation therefore emerges in the Prisoner's Dilemma, and games of common interest converge to mutual defection. For two-action games the classification is complete and rests on an exact Hamiltonian structure shared by both payoff definitions. The structure gives a behavioural test: cycles in the classic Matching Pennies game have amplitude-independent period (isochronous) under logarithmic payoffs, but not under expected utility. Logarithmic evidence integration thus determines which strategic outcomes dynamical selection can reach, wherever learning approximates Bayesian updating.

0.23