CS Paper Radar

Control and Estimation Co-Design via Envelope-Theorem Gradients

Mohammad S. Ramadan, Philip Dinenis, Mihai Anitescu

eess.SY math.OC ·submitted 2026-09-28 18:32 UTC · v1
theory

Abstract

Instead of the sequential plant--control--estimator design pipeline, which is in general not optimal, we propose the Control and Estimation Co-Design (ContEst) framework, which poses the entire system design as a single two-stage problem solved by first-order methods. A key enabler is that the gradient of each inner control/estimation optimal cost with respect to the design parameters is available from the dual variables the inner solver already returns, by the envelope theorem: exactly under mild regularity (a unique inner minimizer), and as a subgradient under more relaxed conditions. That is, no differentiation through the optimizer or its optimality conditions is needed. We first present the framework in a linear quadratic Gaussian (LQG) regime, where the control and estimation costs become two semidefinite programs (SDPs) whose design gradients are read from the duals of the Lyapunov constraints. We extend the results to robust ($H_\infty$) formulation to cover worst-case control and filtering problems. We also present an extension to constrained and nonlinear systems, using an extended Kalman filter (eKF) and a local linearization of the information-state dynamics to yield an approximate but convex model predictive control (MPC) problem whose design gradients are read directly from the costates. We apply our methods to a variety of realistic design examples from different fields and report significant design improvements.

Closest papers

Nearest neighbours by meaning across the whole index.

Optimal input design via Frank-Wolfe

eess.SY math.OC ·2026-09-08 ·Fethi Bencherki, Bruce Lee, Nikolai Matni, Anders Rantzer
similarity 0.66
Abstract · 7 pages, 4 figures

We study optimal input design over a finite horizon for linear dynamical systems. The goal is to minimize a weighted inverse-covariance (information) criterion subject to an energy budget. The set of covariances achievable by causal policies is convex but lacks a tractable explicit description, ruling out projection-based methods. We show that Frank--Wolfe applies naturally: each linear minimization subproblem is a budget-constrained finite-horizon linear quadratic (LQ) problem, solvable by a Riccati recursion and one-dimensional bisection over a Lagrange multiplier. Using smoothness of the objective over the feasible set, we establish an $\mathcal{O}(1/M)$ convergence rate for the objective value, while strong convexity yields an $\mathcal{O}(1/\sqrt{M})$ rate for the iterates. We further extend the framework to input design for system identification with unknown dynamics and adaptive online LQR, and illustrate the approach numerically.

0.28

Dynamic Programming for A Linear-Gaussian Dual Control Problem

eess.SY math.OC ·2026-10-13 ·Ying Wang, Yuchao Li, Håkan Hjalmarsson
similarity 0.60
Abstract

We consider an infinite-horizon discounted dynamic programming (DP) problem for a countable-innovation belief-state model. The model is a surrogate for a linear-Gaussian dual control problem involving an unknown parameter vector and Gaussian observation noise. The surrogate preserves the first two moments of the Gaussian innovation and the resulting posterior second-moment structure. We allow unbounded belief and control spaces and nonnegative stage costs with quadratic growth in the posterior mean and covariance. Using a posterior-moment weight, we show that the Bellman operator is a contraction in a weighted sup-norm space. We characterize the optimal cost as the unique solution of the Bellman equation in this space, establish the existence of optimal stationary policies. We further prove weighted sup-norm convergence of value iteration, policy iteration, and optimistic policy iteration under vanishing approximation errors. The convergence results under vanishing approximation errors rely on general weighted sup-norm contraction properties and are applicable beyond the particular dual control problem considered here.

0.31

Square Root Gauss-Newton iLQR

cs.RO math.OC ·2026-09-17 ·Maximilian Haas-Heger, Jur van den Berg
similarity 0.59
Abstract · 16 pages, 2 figures, ISRR 2026

The iterative Linear Quadratic Regulator (iLQR) is a widely used algorithm for nonlinear trajectory optimization. At each iteration, it solves a local linear-quadratic approximation of the problem via dynamic programming, propagating a quadratic cost-to-go function. If the Hessian of the cost-to-go approximation is positive-semidefinite, one can derive a square root formulation of iLQR that propagates its Cholesky factor instead. This offers significant numerical advantages - much as square root Kalman filters improve upon their conventional counterparts - particularly when iLQR is used within an augmented Lagrangian framework for handling constraints, where large penalties degrade conditioning. Previous square root formulations of iLQR and related algorithms exist, but they are either numerically suboptimal, algorithmically complex, or both. In this paper, we show that the key to an effective square root formulation lies in the Gauss-Newton (weighted least-squares) structure of the cost function: this yields a positive semidefiniteness property that extends beyond the Hessian to the full augmented cost-to-go matrix, and enables a backward pass of remarkable simplicity in which each step reduces to a single QR-decomposition, from which the feedback gain and propagated Cholesky factor are extracted directly.

0.23

A Behavioral approach to Constrained Linear Quadratic Control: Feasibility and Optimality

eess.SY math.OC ·2026-10-29 ·Soutrik Bandyopadhyay, Debasattam Pal, Shubhendu Bhasin
published similarity 0.58
Abstract · Accepted for publication at IEEE Indian Control Conference 2027

This paper develops a constrained linear-quadratic (LQ) optimal control framework within the behavioral approach. Unlike classical formulations that rely on state-input representations, our method treats both the system dynamics and the constraints as trajectory-level objects. The system is modeled through a kernel representation, while inequality constraints are captured using behavioral inequalities defined over infinite-dimensional convex cones. Using quadratic differential forms (QDFs) to encode performance indices, we formulate the constrained LQ problem as a convex variational optimization over trajectories. By employing Gateaux differentials, we derive the stationarity and complementarity conditions and show that they admit a representation as an augmented behavioral system whose unique decomposition into autonomous and controllable sub-behaviors yields a complete parameterization of all optimal constrained trajectories. The resulting conditions constitute a behavioral analogue of the Karush-Kuhn-Tucker (KKT) framework. A numerical example illustrates the effect of inequality constraints on the optimal behavior. The framework provides a general, trajectory-level representation for constrained optimal control and opens avenues for closed-loop and data-driven extensions.

0.41

Safe Output-Regulation Flows for Constrained Optimization via Control Barrier Functions

eess.SY ·2026-11-05 ·Zihao Ren, Bingqi Liu, Lei Wang, Hongye Su, Guodong Shi
similarity 0.57
Abstract

Control-theoretic approaches to constrained optimization have been widely studied by interpreting optimization algorithms as dynamical systems. This paper formulates constrained optimization as an output-regulation problem with state constraints, where equality-constraint residuals are treated as regulated outputs and inequality constraints as state constraints. Proportional--integral feedback is employed to drive the equality-constraint residuals asymptotically to zero, while a control barrier function (CBF) safety filter is used to ensure inequality safety throughout the optimization process. We establish feasibility of the CBF filtering program throughout the inequality-feasible set and local exponential convergence of the resulting flow. Under stronger global conditions, we further prove convergence from an entire connected component of the inequality-feasible set. Since exact implementation of the flow requires solving a CBF quadratic program at each state, we further develop a flow that approximates the CBF solution through multiplier updates. Moreover, we provide a unified formulation encompassing the two proposed flows and projected primal--dual dynamics. Numerical examples demonstrate convergence, safety, and computational tradeoffs of the proposed flows.

0.13

Gradient-Based Co-Design of Systems and Reinforcement Learning Policies with Differentiable Simulators

cs.LG eess.SYmath.OC ·2026-11-14 ·Akhil S. Anand, Xun Shen, Marc Peter Deisenroth, Sebastien Gros
benchmarktheory similarity 0.57
Abstract · 31 pages, 5 figures, 3 tables

Co-design is the problem of jointly optimizing the design of a system and the policy that operates it. Neither can be chosen independently: a system design is only as good as the policy trained for it, and the optimal policy changes with the design. When that policy is learned by reinforcement learning (RL), evaluating a candidate design requires a full policy training run, and the design is usually optimized by black-box search such as Bayesian optimization (BO) or evolution strategies (ES). These methods scale poorly with the design dimension and ignore the differentiable structure of modern simulators and, in particular, the sensitivity of the optimal policy with respect to the design. We instead derive a gradient-based co-design method, where the design gradient decomposes into a direct effect of changing the design at a fixed policy and an indirect effect through the design-dependent optimal policy. The indirect effect requires the sensitivity of the converged policy to the design, which we obtain by applying the implicit function theorem to the stationarity condition of the policy gradient. Based on this sensitivity, we develop a proof-of-concept co-design algorithm with stochastic value gradient (SVG)-style policy optimization, prove that it converges to an approximately stationary design, and demonstrate on a differentiable networked-system benchmark with high-dimensional design spaces that it consistently outperforms BO and ES.

0.33

Antagonistic Control: Foundations, Scalability and Nonlinearity

eess.SY math.OC ·2026-08-23 ·Sribalaji C. Anand, André M. H. Teixeira
similarity 0.56
Abstract · Under review

This paper studies the worst-case impact of constrained control inputs: an input seeks to maximize the average cost of some outputs, measured in the $L_2$ or $L_1$ norm, while remaining bounded in terms of other outputs. This problem template subsumes classical metrics such as the $H_\infty$ norm and the output-to-output gain, and arises in adversarial control, security assessment, and robust control. For linear time-invariant systems, we provide an exact semi-definite program (SDP) when there is a single constraint, and SDPs computing upper bounds when there are multiple constraints. We derive sufficient conditions, in terms of system zeros and relative degrees, under which the worst-case cost is unbounded, together with a constructive closed-loop modification that removes the unboundedness. From a security standpoint, unbounded values reveal structural limitations in detecting certain attack inputs. For positive systems, we provide scalable formulations whose complexity grows linearly in the state dimension: a scalable SDP for quadratic costs, and an exact linear program for linear costs. The results extend to nonlinear polynomial systems via a sum-of-squares program. We illustrate the results with numerical examples.

0.20

Sparse Experimental Design for Nonsmooth Estimators via Bilevel Optimization

eess.SP ·2026-10-01 ·Antonio G. Marques, Samuel Rey
code/data similarity 0.56
Abstract

Optimal experimental design (OED) decides which measurements to acquire for downstream estimation. Classically, this is done by optimizing an information criterion derived from a linear-Gaussian model, for which the estimator is available in closed form. Many modern estimators such as Lasso, elastic net, or total-variation-based estimators, however, are nonsmooth and lack a closed-form solution. To extend OED to these scenarios, we recast the problem as what it implicitly is: a bilevel program whose lower level computes the deployed nonsmooth estimator and whose upper level scores its validation prediction risk. Rather than fixing the number of measurements in advance, we charge each continuous acquisition weight a concave sparsity price. As a result, how many and which measurements to keep are both outcomes of the optimization. A measurement survives only if its estimator-aware value exceeds its price, and progressively increasing the price yields a sequence of designs that explores the trade-off between measurement cardinality and estimation accuracy. Using a value-function penalty reformulation, we develop a single-loop proximal-gradient algorithm that avoids differentiating the nonsmooth solution map and establish its convergence to an (approximate) stationary point. Experiments on synthetic sparse recovery and image reconstruction demonstrate the benefits of the proposed method.

0.38