CS Paper Radar

No Spurious Local Minima in Full-Order Linear Quadratic Gaussian Control

Yang Zheng, Yujie Tang

eess.SY math.OC ·submitted 2026-09-26 18:58 UTC · v1

Abstract

This paper studies the nonconvex optimization landscape of Linear Quadratic Gaussian (LQG) control under direct state-space parameterization. Although the LQG cost may possess suboptimal stationary points, whether it admits suboptimal local minima has remained open. We answer this question negatively: Every local minimum of the full-order LQG cost is globally optimal. More generally, at any controller order, every local minimum corresponding to an uncontrollable or unobservable realization attains the globally optimal LQG cost over all stabilizing policies of arbitrary orders. We further show that any suboptimal policy can be augmented with decoupled stable controller states, so that the augmented policy admits a direction of negative curvature with probability one. Consequently, any suboptimal stationary point in full-order LQG cost can be converted into a strict saddle with probability one. One key proof idea connects the state-space Hessian to a frequency-domain global-optimality condition derived from the Youla parameterization. These results reveal a benign local-minimum landscape despite the presence of suboptimal stationary points.

Closest papers

Nearest neighbours by meaning across the whole index.

First-Order Model-Free Policy Gradient for Undiscounted LQR with Global Optimality and Convergence Guarantees

eess.SY math.OC ·2026-11-16 ·Vipul K Sharma, S Sivaranjani
similarity 0.63
Abstract

We provide a first-order model-free policy gradient (PG) framework for learning undiscounted infinite-horizon linear quadratic regulator (LQR) policies with global optimality and convergence guarantees. We first construct a stochastic wrapper policy class parameterized by the LQR gain to enable exploration around a deterministic linear control law. We analyze the additional cost induced by exploration over infinite-horizon, and establish a weak gradient dominance condition that characterizes global optimality under exploration. We then construct a model-free first-order PG algorithm using sampled trajectory information, without identifying the dynamics or approximating a value function. Finally, we establish finite-time convergence guarantees, providing the sufficient number of samples required for near-global optimality with our proposed PG algorithms.

0.21

Minimax optimal dual control of positive systems: an exact solution for scalar input-sign uncertainty

eess.SY math.OC ·2026-08-15 ·Fethi Bencherki, Tomas J. Meijer, Anders Rantzer
similarity 0.61
Abstract

While recent advances in minimax dual control have led to exact solutions for uncertain general linear time-invariant systems as well as (sub)optimal dual controllers, corresponding results for linear positive systems are still lacking. This paper aims to fill this gap and thereby pave the way toward scalable dual control algorithms. We study the general minimax optimal dual control problem for positive linear systems with unknown dynamics and reformulate it as a standard zero-sum dynamic game. By allowing randomized control inputs, we solve the corresponding Bellman equation exactly for the scalar case with sign uncertainty in the input. This yields an implicit dual control policy that is optimal both in terms of cost and $\ell_1$-gain. The optimal dual policy uses exploration in a specific region of the hyperstate space to conduct optimal probing. Outside this exploration regime, the controller reduces to a deterministic certainty equivalence policy, indicating that sufficient information has been obtained to identify the correct input direction. In addition, these results allow us to analyze fundamental limitations of minimax dual control for positive systems and provide a foundation for more general dual control problems for positive systems for future work.

0.04

Online Constrained Control of Storage Systems via Simplex Disturbance-Action Policies

eess.SY ·2026-09-07 ·Kamiar Asgari, Michael J. Neely
theory similarity 0.60
Abstract

We study online control of a scalar storage system with nonnegative adversarial resource arrivals and state-dependent action constraints. The adversarial, time-varying cost depends on both state and action. At each time, the controller selects a feasible action before the current resource arrival and cost are revealed. To handle the coupling between feasibility and online learning, we introduce Simplex Disturbance-Action Control (SDAC), whose policies have $H$ nonnegative parameters summing to at most one. Every fixed SDAC policy is feasible by construction. Our online SDAC controller updates these parameters using entropic online mirror descent while preserving feasibility along the time-varying trajectory. We prove a regret bound of $O\!\left(\sqrt{T\log(H+1)}\right)$ relative to the best fixed SDAC policy and a minimax lower bound showing that the dependence on $T$ is tight. We also introduce infinite-memory SDAC policies, which include every feasible fixed-fraction policy. SDAC policies approximate this class with an error that decreases geometrically with $H$. For a suitable choice $H=Θ(\log T)$, the resulting regret against infinite-memory SDAC policies is $O\!\left(\sqrt{T\log\log T}\right)$. The framework applies to energy-harvesting batteries and other storage-constrained resource systems.

0.49

Square Root Gauss-Newton iLQR

cs.RO math.OC ·2026-09-17 ·Maximilian Haas-Heger, Jur van den Berg
similarity 0.59
Abstract · 16 pages, 2 figures, ISRR 2026

The iterative Linear Quadratic Regulator (iLQR) is a widely used algorithm for nonlinear trajectory optimization. At each iteration, it solves a local linear-quadratic approximation of the problem via dynamic programming, propagating a quadratic cost-to-go function. If the Hessian of the cost-to-go approximation is positive-semidefinite, one can derive a square root formulation of iLQR that propagates its Cholesky factor instead. This offers significant numerical advantages - much as square root Kalman filters improve upon their conventional counterparts - particularly when iLQR is used within an augmented Lagrangian framework for handling constraints, where large penalties degrade conditioning. Previous square root formulations of iLQR and related algorithms exist, but they are either numerically suboptimal, algorithmically complex, or both. In this paper, we show that the key to an effective square root formulation lies in the Gauss-Newton (weighted least-squares) structure of the cost function: this yields a positive semidefiniteness property that extends beyond the Hessian to the full augmented cost-to-go matrix, and enables a backward pass of remarkable simplicity in which each step reduces to a single QR-decomposition, from which the feedback gain and propagated Cholesky factor are extracted directly.

0.23

Weak Convexity and Proximal Bundle Methods for Nonsmooth Policy Optimization in Robust Control

eess.SY math.OC ·2026-09-05 ·Yuto Watanabe, Feng-Yi Liao, Yang Zheng
similarity 0.59
Abstract · 36 pages, 15 figures

We study policy optimization for discrete-time robust $\mathcal{H}_\infty$ control with static output-feedback, and present the first feasibility-preserving algorithm with a deterministic, non-asymptotic complexity guarantee. This problem naturally leads to a nonsmooth and nonconvex optimization over the set of stabilizing feedback gains. We first establish several structural properties of the $\mathcal{H}_\infty$ cost. In particular, we show that the cost is weakly convex on every convex subset of a sublevel set. For the state-feedback case, we further establish a weak Polyak--Łojasiewicz inequality, which ensures that every stationary point is globally optimal. Building on these properties, we develop a proximal bundle method for $\mathcal{H}_\infty$ policy optimization. The proposed method can be viewed as an implementable approximation of the proximal point method and uses only function value and subgradient information. We show that all iterates remain stabilizing and establish a deterministic non-asymptotic complexity bound of $\mathcal{O}(\max\{η^{-4},ε^{-2}\})$ for finding an $(η,ε)$-stationary point. Numerical experiments illustrate our theoretical results.

0.30

Inverse Linear Quadratic Gaussian Games: Constrained Setting and Transferability

eess.SY ·2026-09-24 ·Kai Ren, Maryam Kamgarpour
similarity 0.59
Abstract

This work addresses finite-horizon inverse linear quadratic Gaussian games. In a constrained setting, we characterize the set of cost parameters and optimal dual values that generate a given generalized Nash equilibrium, and we propose an algorithm to compute these parameters. In an unconstrained setting, we address transferability, namely, we bound the cost value perturbation between two policies: one induced by the identified cost parameters, the other by the expert parameters, under a set of different dynamics. This cost value perturbation scales linearly with the deviations in the dynamics and the identified cost parameters. Through numerical simulations, we show that, in a constrained setting, our algorithm identifies the cost parameters and dual values that can reproduce the policy and trajectories corresponding to the observed generalized Nash equilibrium. In an unconstrained setting, we show with a traffic simulation and real-robot experiments that the identified cost parameters can be used to control sufficiently close dynamics, with performance degrading linearly with the deviations in the dynamics and the identified cost parameters.

0.33

Policy Iteration for Domain Randomized Linear Quadratic Systems

eess.SY math.OC ·2026-09-04 ·Abbas Pasdar, Farnaz Adib Yaghmaie
published similarity 0.58
Abstract · 8 pages, 2 figures, accepted for CDC 2026 | Honolulu, Hawaii, USA

In this work, we study policy optimization under domain randomization for linear quadratic control, focusing on learning a single state-feedback controller that minimizes the average cost across systems with uncertain dynamics. We propose a policy iteration algorithm with a step-size rule that preserves stability across all sampled systems at each iteration. We show that the method yields monotonic improvement of the sample-average objective and that a stabilizing step size always exists. Under standard smoothness assumptions, the iterates converge subsequentially to stationary points, and under a gradient-dominance condition, we obtain a global linear convergence rate.

0.65

Reinforcement learning for constrained stochastic linear--quadratic control in continuous time

cs.LG math.OC ·2026-10-26 ·Weiping Wu, Qitong Pan, Xun Li, Peng Wu
similarity 0.57
Abstract

We study continuous-time stochastic linear--quadratic control with a scalar state and nonnegative control constraints. With quadratic running costs, randomized exploration can change the optimal mean feedback. We introduce an auxiliary problem without entropy in which the conditional policy mean is nonnegative and the covariance is prescribed in proportion to the squared state. Moment reduction and state separation yield two regional Riccati equations and characterize the finite-horizon value function and optimal conditional mean. We give a sufficient control--cost compatibility condition under which this mean coincides with the optimal feedback of the original constrained problem. For time-invariant coefficients, we characterize the undiscounted infinite-horizon optimum and establish mean square asymptotic stability under the stated assumptions. For unknown dynamics, we develop model-free actor--critic schemes for finite and infinite horizons. Numerical experiments compare the learned policies with their analytical targets and examine the effect of exploration on the optimal mean.

0.25