CS Paper Radar

General-Sum Linear Regulator Games for Positive Systems

Alba Gurpegui, Monika Tomar, Takashi Tanaka

eess.SY math.OC ·submitted 2026-08-24 01:59 UTC · v1
publishedtheory

Abstract

This paper studies a continuous-time general-sum non-cooperative game with linear costs, positive linear system dynamics, and elementwise linear input constraints. In the finite-horizon case, we present a verification theorem characterizing feedback Nash equilibria, in terms of absolutely continuous solutions of a coupled system of vector-valued ordinary differential equations, realized by time-varying feedback laws. Unlike linear-quadratic differential games, whose Riccati-based equilibria scale quadratically with the state dimension, this formulation scales linearly. However, the resulting piecewise-constant feedback saturates between its constraint bounds rather than varying smoothly, and additional mathematical challenges arise when characterizing the solutions of the differential equations, which are generally discontinuous due to the switching nature of the feedback gains. In this work, we study the case where switching occurs only at isolated time instants. In the infinite-horizon case, under stabilizability assumptions, the equilibrium is characterized by coupled vector-valued algebraic equations. For this game, we propose iterative methods to compute both finite and infinite-horizon equilibria. The approach is illustrated through a large-scale pollution game.

Comment
6 pages and 3 figures. Accepted for presentation at the 65th IEEE Conference in decision and control 2026 (CDC)

Closest papers

Nearest neighbours by meaning across the whole index.

Differential Games for Compositional Handling of Competing Control Tasks

cs.MA ·2026-09-01 ·Joshua Shay Kricheli
benchmarkcode/data similarity 0.64
Abstract · Thesis submitted in partial fulfillment of the requirements for the Master of Sciences degree, The Department of Computer Science, The Facul…

We introduce a novel Divide and Conquer control design methodology leveraging differential games in single-agent, multi-objective dynamical systems. The proposed framework associates each control objective with a virtual input and establishes a non-cooperative, finite or infinite horizon differential game among representative players. Each player optimizes a distinct virtual cost function tailored to its specific goal, the full system state, and the other virtual inputs, while accounting for the remaining players' optimal policies. By establishing a Nash Equilibrium for this game, we synthesize a composite controller that achieves a stable balance across competing objectives, providing control engineers with an intuitive and modular framework for parameter re-tuning throughout the design cycle. We provide formal mathematical derivations for both continuous-time and discrete-time dynamical systems, targeting large-scale single-agent applications where complex, dynamically conflicting control objectives make global weighting intractable. To demonstrate the methodology, we developed an open-source Python package implementing a novel numerical algorithm for solving Coupled Algebraic Riccati Equations arising in infinite-horizon differential games. We evaluate the approach on two benchmark case studies: an inverted pendulum on a cart and a non-linear hierarchically controlled quadrotor. The resulting closed-loop performance is compared against the classical Linear Quadratic Regulator (LQR) across various transient and steady-state control metrics, demonstrating superior trajectory tracking and robust multi-objective regulation.

0.29

Infinite-Horizon Inverse Linear-Quadratic Differential Games with State- and Control-Dependent Noise

eess.SY ·2026-08-18 ·Lucas Günther, Karl Handwerker, Felix Thömmes, Balint Varga, Sören Hohmann
similarity 0.61
Abstract

This paper presents a method to solve the inverse problem for N-player infinite-horizon linear-quadratic (LQ) differential games with state- and control-dependent noise. For this stochastic setting, we derive necessary and sufficient conditions for linear feedback Nash equilibria, which take the form of coupled stochastic algebraic Riccati equations. We then derive a kernel representation of these equations to explicitly characterize the set of all cost function parameter combinations across players that are consistent with observed equilibrium trajectories, thereby solving the associated inverse problem. Numerical results illustrate the approach and confirm the theoretical findings, highlighting the inherent ambiguity of the inverse problem.

0.44

An α-potential linear approximation for LQ differential games

eess.SY math.OC ·2026-11-18 ·Balint Varga, Gergely Lorant Kiss, Andreas Schwung
similarity 0.60
Abstract

Feedback Nash equilibria of infinite-horizon linear-quadratic (LQ) differential games are characterized by coupled algebraic Riccati equations (cAREs), whose numerical solution generally relies on iterative algorithms and can be highly sensitive to their initialization. Motivated by the $α$-potential-game framework, this letter proposes an $α$-potential linear approximation for constructing a systematic, game-dependent initialization. For an N-player LQ game, a reference potential game is constructed. Its equilibrium requires solving only one standard algebraic Riccati equation. The original cAREs are subsequently linearized around this reference solution, yielding a coupled linear system whose solution provides player-specific corrections and hence an approximate feedback Nash equilibrium. The resulting approximation is used to initialize three standard cARE solvers. Numerical experiments on $10^4$ randomly generated two-player games show that the proposed initialization reduces the mean number of iterations. Further experiments for $N=2,\ldots,13$ show that the reduction in iteration count persists as the number of players increases.

0.28

Metric-Based Equilibrium Selection in Noncooperative Differentiable Games

cs.GT eess.SY ·2026-10-05 ·Karan Mahesh, Runyu Zhang, Michael R. Benjamin, Gioele Zardini
similarity 0.60
Abstract · 8 pages, 2 figures

Metric conditioning can change which equilibrium attracts learning dynamics without moving the equilibria of a differentiable game. We study how to use state-dependent symmetric positive-definite (SPD) metrics to select among known equilibria by keeping a designated equilibrium attracting while making a rival unstable. Building on classical results on matrix stability under positive-definite multiplication, we characterize when such destabilization is possible and show how restricting the metric to act independently on each player limits the stability changes that can be achieved at differential Nash equilibria. We then combine a stabilizing metric at the selected equilibrium with a destabilizing metric at the rival through smooth interpolation, yielding a single metric field that preserves every equilibrium of the game while assigning the desired local stability types. We also derive a step-size condition for implementing the method in discrete time. The approach is validated on a continuous-commitment stag hunt and an entropy-regularized iterated prisoner's dilemma, where it selects the desired equilibrium in both settings.

0.28

Critic-Free Policy Iteration for Continuous-Time Zero-Sum Games: A Policy-Space Riccati Approach

eess.SY ·2026-09-10 ·Jiacheng Wu, Yang Zhu, Hongye Su
similarity 0.60
Abstract

This paper develops a critic-free policy iteration (PI) method for continuous-time linear zero-sum games. The central idea is to characterize the saddle-point policies directly in the joint policy space, rather than treating the quadratic value matrix as an iterative variable. A policy game Riccati equation (PGRE) is introduced whose unknowns are policy gains only. Its solutions are shown to be in one-to-one correspondence with the symmetric solutions of the game algebraic Riccati equation. Based on a stabilizing anchor, PI is performed directly in the actor space. The actor-space Jacobian is nonsingular at every stabilizing policy, and the resulting policy sequence coincides with that of simultaneous PI. For unknown dynamics, a data-driven algorithm uses a single batch of data and nullspace projection of endpoint increments to eliminate the value matrix, yielding an actor-only regression. A necessary and sufficient rank condition for unique policy recovery is established and shown to be iteration-invariant, demonstrating that critic identifiability is unnecessary. A power systems frequency-regulation example verifies convergence and policy recovery, while scalability tests demonstrate substantial reductions in computational and memory requirements.

0.31

Two-Player Zero-Sum Hybrid Games with Set-Valued Dynamics

eess.SY math.OC ·2026-11-03 ·Santiago J. Leudo, Francesco Ferrante, Ricardo G. Sanfelice
similarity 0.59
Abstract · 31 pages, 3 figures

In this paper, we formulate a two-player zero-sum game under dynamic constraints defined in terms of a hybrid inclusion. The game consists of a min-max problem involving a cost functional that depends on the actions and corresponding (potentially non-unique) solutions, defined as functions of hybrid time and, hence, can flow or jump. We present sufficient conditions given in terms of Hamilton-Jacobi-Isaacs-like equations to establish a bound and to exactly evaluate the worst-case cost under the optimal strategy. Under additional conditions, we show that the proposed optimal state-feedback laws render a set of interest pre-asymptotically stable for the resulting hybrid closed-loop system. The results are illustrated in an application with sporadic jumps and linear quadratic hybrid dynamics.

0.38

Infinite Horizon Mean-Field Terminal Value Problem

eess.SY math.OC ·2026-09-11 ·Uğur Aydın, Tamer Başar, Naci Saldi
similarity 0.58
Abstract · 39 pages, 2 figures

We introduce a class of finite- and infinite-horizon discrete-time control problems for studying macroscale systems in discrete time, which we refer to as mean-field terminal value problems (MFTVPs). An MFTVP takes a terminal \(Q\)-function as input and, starting from this terminal \(Q\)-function, seeks an indefinite backward evolution of optimal policies and state measures, as required for an infinite-horizon non-stationary mean-field equilibrium (MFE) of a discrete-time mean-field game (MFG). We study the existence and uniqueness properties of this problem. As an application, using the uniqueness properties of MFTVPs, we show that, in the average-cost setting (and in the discounted setting for sufficiently small discount factors), MFTVPs can be used as an intermediate step to develop approximation schemes between finite-horizon MFEs and average-cost MFEs (respectively, stationary MFEs). In contrast to approaches that use infinite-horizon MFGs as intermediate problems, MFTVPs can be employed when the discount factor is sufficiently small (or, in the average-cost setting, when the drift factor is sufficiently small), together with a regularizer, to approximate a stationary MFE. Assuming uniqueness of the solution of a MFTVP, the restrictions we impose on the Lipschitz parameters of the system are weaker than the contraction conditions available for regularized stationary MFGs, and our approximation scheme remains applicable even in the presence of multiple stationary MFEs.

0.30

Reinforcement learning for constrained stochastic linear--quadratic control in continuous time

cs.LG math.OC ·2026-10-26 ·Weiping Wu, Qitong Pan, Xun Li, Peng Wu
similarity 0.57
Abstract

We study continuous-time stochastic linear--quadratic control with a scalar state and nonnegative control constraints. With quadratic running costs, randomized exploration can change the optimal mean feedback. We introduce an auxiliary problem without entropy in which the conditional policy mean is nonnegative and the covariance is prescribed in proportion to the squared state. Moment reduction and state separation yield two regional Riccati equations and characterize the finite-horizon value function and optimal conditional mean. We give a sufficient control--cost compatibility condition under which this mean coincides with the optimal feedback of the original constrained problem. For time-invariant coefficients, we characterize the undiscounted infinite-horizon optimum and establish mean square asymptotic stability under the stated assumptions. For unknown dynamics, we develop model-free actor--critic schemes for finite and infinite horizons. Numerical experiments compare the learned policies with their analytical targets and examine the effect of exploration on the optimal mean.

0.25