CS Paper Radar

Two-Player Zero-Sum Hybrid Games with Set-Valued Dynamics

Santiago J. Leudo, Francesco Ferrante, Ricardo G. Sanfelice

eess.SY math.OC ·submitted 2026-11-03 14:50 UTC · v1

Abstract

In this paper, we formulate a two-player zero-sum game under dynamic constraints defined in terms of a hybrid inclusion. The game consists of a min-max problem involving a cost functional that depends on the actions and corresponding (potentially non-unique) solutions, defined as functions of hybrid time and, hence, can flow or jump. We present sufficient conditions given in terms of Hamilton-Jacobi-Isaacs-like equations to establish a bound and to exactly evaluate the worst-case cost under the optimal strategy. Under additional conditions, we show that the proposed optimal state-feedback laws render a set of interest pre-asymptotically stable for the resulting hybrid closed-loop system. The results are illustrated in an application with sporadic jumps and linear quadratic hybrid dynamics.

Comment
31 pages, 3 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Minimax optimal dual control of positive systems: an exact solution for scalar input-sign uncertainty

eess.SY math.OC ·2026-08-15 ·Fethi Bencherki, Tomas J. Meijer, Anders Rantzer
similarity 0.60
Abstract

While recent advances in minimax dual control have led to exact solutions for uncertain general linear time-invariant systems as well as (sub)optimal dual controllers, corresponding results for linear positive systems are still lacking. This paper aims to fill this gap and thereby pave the way toward scalable dual control algorithms. We study the general minimax optimal dual control problem for positive linear systems with unknown dynamics and reformulate it as a standard zero-sum dynamic game. By allowing randomized control inputs, we solve the corresponding Bellman equation exactly for the scalar case with sign uncertainty in the input. This yields an implicit dual control policy that is optimal both in terms of cost and $\ell_1$-gain. The optimal dual policy uses exploration in a specific region of the hyperstate space to conduct optimal probing. Outside this exploration regime, the controller reduces to a deterministic certainty equivalence policy, indicating that sufficient information has been obtained to identify the correct input direction. In addition, these results allow us to analyze fundamental limitations of minimax dual control for positive systems and provide a foundation for more general dual control problems for positive systems for future work.

0.04

General-Sum Linear Regulator Games for Positive Systems

eess.SY math.OC ·2026-08-24 ·Alba Gurpegui, Monika Tomar, Takashi Tanaka
publishedtheory similarity 0.59
Abstract · 6 pages and 3 figures. Accepted for presentation at the 65th IEEE Conference in decision and control 2026 (CDC)

This paper studies a continuous-time general-sum non-cooperative game with linear costs, positive linear system dynamics, and elementwise linear input constraints. In the finite-horizon case, we present a verification theorem characterizing feedback Nash equilibria, in terms of absolutely continuous solutions of a coupled system of vector-valued ordinary differential equations, realized by time-varying feedback laws. Unlike linear-quadratic differential games, whose Riccati-based equilibria scale quadratically with the state dimension, this formulation scales linearly. However, the resulting piecewise-constant feedback saturates between its constraint bounds rather than varying smoothly, and additional mathematical challenges arise when characterizing the solutions of the differential equations, which are generally discontinuous due to the switching nature of the feedback gains. In this work, we study the case where switching occurs only at isolated time instants. In the infinite-horizon case, under stabilizability assumptions, the equilibrium is characterized by coupled vector-valued algebraic equations. For this game, we propose iterative methods to compute both finite and infinite-horizon equilibria. The approach is illustrated through a large-scale pollution game.

0.26

Competitive One-Step-Ahead Control of Friedkin--Johnsen Networks: Potential Games, Stability, and the Price of Competition

eess.SY ·2026-08-27 ·Gabriel Gentil, Amit Bhaya
similarity 0.58
Abstract · 11 pages, 2 figures. Submitted to IEEE Transactions on Control of Network Systems

This paper studies competitive one-step-ahead control of Friedkin-Johnsen networks with overlapping player influence. The one-step interaction is an exact potential game with a unique Nash equilibrium obtained from a symmetric positive-definite system. Sequential best-response sweeps converge for every frozen network state, parallel sweeps obey an exact Jacobi condition (and always converge with two players), and one-sweep implementations require an augmented state-action stability test. For marginal networks, a signed left-right damping condition is sufficient for exact-equilibrium stability and becomes a sharp first-order instability test when its sign is reversed. A resolvent identity clarifies the feedback geometry, while a control-aware centrality identifies the goal conflicts that matter most. We characterize attainable equilibria under unconstrained, convex, and sparse goal restrictions and give a closed form for the same-state welfare loss caused by competition. Numerical examples verify the stability thresholds, geometry, and welfare predictions.

0.55

Last-Iterate Convergence of Policy Dynamics in Zero-Sum Networked Separable Markov Games

cs.GT ·2026-09-08 ·Zailin Ma
similarity 0.57
Abstract

Solving Nash equilibria for general multi-player Markov games is computationally intractable, while two-player zero-sum Markov games admit fast last-iterate policy-optimization methods. Zero-sum networked separable Markov games occupy an important middle ground: they retain global multi-player competition structure through pairwise interactions, while preserving computational tractability of Nash equilibria (NE) in the finite-horizon setting. Existing algorithms for this class either proceed through equilibrium-collapse arguments for a simplified setting where a single controller determines the transition probability, or backward dynamic programming that relies on equilibrium solvers at each stage. However, the design and analysis of direct policy-update approaches remain inadequate. To address this issue, we propose the entropy-regularized optimistic multiplicative weights update (ER-OMWU), a complementary single-loop policy dynamic that updates players' policies symmetrically and returns an approximate NE in the last iteration. We provide the first last-iterate convergence analysis of policy dynamics in the games of interest: after $\widetilde O\left(1/ε\right)$ iterations, the returned policy is an $ε$-approximate Nash equilibrium. The result preserves the near-linear convergence rate achieved by policy optimization in two-player zero-sum Markov games, but extends the policy-dynamics viewpoint to a more complicated but structured multi-player setting.

0.20

Exact Decomposition of Value Functions for Two-Player Games in Hamilton-Jacobi Reachability

eess.SY ·2026-08-27 ·Dylan Hirsch, William Sharpless, Sylvia Herbert
similarity 0.56
Abstract

Hamilton-Jacobi reachability (HJR) is an important framework in safe control theory. HJR provides theoretical tools and numerical methods to obtain value-functions for tasks involving target-reaching and obstacle-avoidance. A recent work proposed an algebraic framework to scale to complex, composite tasks via decomposing the value functions of the composite tasks into value functions for the fundamental tasks traditionally studied in HJR, all in the one-player setting. Many of these decomposition results, however, do not directly translate to the two-player setting. In this technical note, we nevertheless show that one of the previous work's key results for analyzing various tasks involving reaching multiple targets while obeying constraints still holds in the two-player setting, so long as a critical monotonicity assumption is satisfied. In particular, we detail the analogous result and its proof (which structurally differs from the one-player case), we show via a counter-example what can go wrong if this monotonicity assumption is not satisfied, and we show how the analysis of a variety of tasks can be performed using this result. Such tasks include sequential and any-order multi-target-reaching tasks under constraints. We note that, whereas the prior work considered discrete-time, infinite-horizon tasks, which are more standard in reinforcement learning, we here consider continuous-time, finite-horizon tasks, which are more standard in HJR.

0.24

Homological Barriers to Stable Learning of First-Order Nash Equilibria

cs.GT cs.LG ·2026-11-08 ·Ashkan Soleymani, Gabriele Farina, Patrick Jaillet, Georgios Piliouras
published similarity 0.56
Abstract · The Fortieth Annual Conference on Neural Information Processing Systems (NeurIPS) 2026

We identify a homological barrier to stable learning of local Nash equilibria, also known as first-order Nash equilibria (FONE), in constrained non-concave differentiable games. We consider continuous deterministic learning dynamics that fix every FONE point and aim to attract all initial conditions to the full FONE set. We show that this convergence guarantee is impossible when the target set has nontrivial positive-degree cohomology, such as that arising from a loop, and the state space is compact and contractible. We exhibit this barrier in a simple non-concave quadratic zero-sum game on a box whose exact FONE set consists of a diagonal six-edge loop and an isolated point. Consequently, no such dynamics can make the full FONE set a stable global attractor. We further show that degree two is the minimal polynomial degree at which this barrier can arise and extend the construction to obtain nontrivial homology in arbitrarily high degrees. Finally, we establish that the barrier persists for sufficiently small approximate-FONE targets under small perturbations, yielding an open, positive-volume family of quadratic zero-sum games with the same homological barrier.

0.41

Infinite Horizon Mean-Field Terminal Value Problem

eess.SY math.OC ·2026-09-11 ·Uğur Aydın, Tamer Başar, Naci Saldi
similarity 0.56
Abstract · 39 pages, 2 figures

We introduce a class of finite- and infinite-horizon discrete-time control problems for studying macroscale systems in discrete time, which we refer to as mean-field terminal value problems (MFTVPs). An MFTVP takes a terminal \(Q\)-function as input and, starting from this terminal \(Q\)-function, seeks an indefinite backward evolution of optimal policies and state measures, as required for an infinite-horizon non-stationary mean-field equilibrium (MFE) of a discrete-time mean-field game (MFG). We study the existence and uniqueness properties of this problem. As an application, using the uniqueness properties of MFTVPs, we show that, in the average-cost setting (and in the discounted setting for sufficiently small discount factors), MFTVPs can be used as an intermediate step to develop approximation schemes between finite-horizon MFEs and average-cost MFEs (respectively, stationary MFEs). In contrast to approaches that use infinite-horizon MFGs as intermediate problems, MFTVPs can be employed when the discount factor is sufficiently small (or, in the average-cost setting, when the drift factor is sufficiently small), together with a regularizer, to approximate a stationary MFE. Assuming uniqueness of the solution of a MFTVP, the restrictions we impose on the Lipschitz parameters of the system are weaker than the contraction conditions available for regularized stationary MFGs, and our approximation scheme remains applicable even in the presence of multiple stationary MFEs.

0.30

Hamilton-Jacobi Reachability for Hybrid Systems: Unified Goal-Driven Control with Safety Guarantees

cs.RO eess.SY ·2026-09-15 ·Javier Borquez, Shuang Peng, Somil Bansal
published similarity 0.54
Abstract

Hybrid dynamical systems provide a powerful modeling framework for robotic systems, particularly in contact-rich environments. However, ensuring safety and performance in such systems remains challenging due to the intricate coupling between continuous dynamics and discrete mode transitions. In this work, we extend classical Hamilton-Jacobi (HJ) reachability analysis, a formal verification method for continuous-time nonlinear systems, to hybrid dynamical systems. Our framework characterizes safe sets for hybrid systems through a generalized value function defined over both discrete and continuous states while accounting for control constraints and model uncertainty. We additionally provide a numerical algorithm to compute this value function. Building on these safe sets, we propose two different mechanisms to integrate performance objectives. First, we introduce a hybrid least-restrictive safety filter that intervenes on both the discrete and continuous components of a nominal controller only when necessary to avoid unsafe states, thereby preserving nominal behavior whenever possible. Second, we formulate and compute hybrid backward reach-avoid tubes, enabling the simultaneous enforcement of safety and goal-reaching behavior, an extension not previously addressed within hybrid HJ reachability. This enables the synthesis of continuous and discrete control policies that guarantee both safety and task completion. We validate our framework through simulation studies and real-world experiments on a quadrupedal robot, demonstrating its effectiveness in hybrid mode planning and safety-critical applications.

0.60