CS Paper Radar

Necessary and Sufficient Conditions for Adaptive Stabilizability of Continuous-Time Markov Jump Linear Stochastic Systems

Ziwei Guo

eess.SY math.OC ·submitted 2026-10-25 10:28 UTC · v1

Abstract

The purpose of this paper is to provide a mathematically rigorous solution to a well-known problem in stochastic adaptive control, i.e., to find the weakest possible structural condition for global stabilizability of continuous-time Markov jump linear stochastic systems with known candidate modes but with an unobservable Markov parameter process. Under some regularity assumptions, it is shown that the necessary and sufficient condition for global adaptive stabilizability hinges on the solvability of the same coupled Riccati equations as in the non-adaptive case. This result highlights a fundamental difference between the continuous-time case considered here and the discrete-time case studied earlier by Xue and Guo (2001), in which the coupled Riccati equations for adaptive and non-adaptive stabilizability are different.

Comment
19 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Feedback Stabilization for Sampled Linear Systems with Control-linear Noise

eess.SY math.OC ·2026-09-24 ·Cynthia Cheng, Xudong Chen
similarity 0.63
Abstract

In this paper, we consider linear stochastic systems with control-linear noise and periodically sampled measurements. We address the problem of feedback stabilization in the mean-square sense. The main contribution of the paper is to provide a necessary and sufficient condition for feedback stabilization. In particular, we relate feedback stabilizability of the stochastic system to the existence of a fixed point of a Riccati-type algebraic equation and, further, to the existence of a solution to an infinite-horizon optimal control problem.

0.22

Control of Markov Jump Linear Systems with Uncertain Lumpable Cluster Observations

eess.SY math.OC ·2026-10-01 ·Carlos A. F. Persiani, Ram Padmanabhan, Melkior Ornik, Marco H. Terra
similarity 0.61
Abstract · 8 pages, 2 figures

In this paper, we consider the control of Markov jump linear systems when the active system mode is not exactly known. We rather assume that this true mode is only known to belong to an observed cluster of modes, making both the dynamics and the associated transition probabilities uncertain. We construct a min-max optimal control problem in this setting, with the objective of robust regulation. Using results in regularized least mean-square optimization, we describe how the solution of this problem can be reduced to a Riccati recursion, such that the controller depends only on knowledge of which cluster the mode lies in. We subsequently establish convergence of the recursion and stability of the resulting closed-loop system. A numerical example illustrates the effectiveness of the proposed methodology.

0.29

On Stochastic Continuous-Time Linear System Identification for Adaptive Stabilization

cs.LG stat.ML ·2026-11-02 ·Reza Sadeghi Hafshejani, Mohamad Kazem Shirani Faradonbeh
similarity 0.58
Abstract

System identification studies the canonical problem of reliably learning system-dynamics from state and input trajectories. One of its important applications consists of estimating a feedback gain matrix that can stabilize an unknown system. This paper focuses on the analysis of such adaptive stabilization schemes for multivariate linear time-invariant state-space models. The system instability practically renders trajectories short, i.e., the observed data is notably limited. So, it is crucial to tightly consider effects of \emph{all} involved quantities including length and number of trajectories, dimension, controllability Gramian, and signal-to-noise ratio. We put forward a simple, practical, and reliable relationship between the chance that adaptive stabilization (based on system identification) fails to stabilize the system, in terms of all the above-mentioned quantities. Our findings show that despite exponential effects of all other quantities on failure probability, such effects do not hold for signal-to-noise ratio. More specifically, this study shows that the strength of the excitation signal helps stabilization only \emph{logarithmically}; a very important phenomenon that we explain under the term \emph{signal saturation}.

0.29

On asymptotic stability of the time-varying Kalman filter for unstabilizable linear systems: an optimization perspective

eess.SY ·2026-09-16 ·James B. Rawlings, Titus Quah, Matthias A. Müller
similarity 0.53
Abstract

This paper establishes the necessary and sufficient conditions for asymptotic stability of the time-varying Kalman filter applied to a linear time invariant system with semidefinite initial state covariance and positive definite process and measurement noise. Rather than analyze the discrete Riccati equation as in the classic literature, the equivalent state smoothing optimization problem is stated and all results are established using properties of this optimization problem. A Lyapunov-like function, termed a modified $Q$-function is derived and used for this analysis. This optimization approach removes the need for the classic but cumbersome Riccati iteration algebra and provides better generalization and application for nonlinear systems.

0.22

Deep Pontryagin Learning for Continuous-Time Stochastic Control with Jumps

cs.LG eess.SYmath.OC ·2026-10-19 ·Jean-Loup Dupret, Patrick Cheridito
theory similarity 0.51
Abstract

We develop a deep Pontryagin learning framework for high-dimensional continuous-time stochastic control problems that naturally accommodates jump-diffusions. In contrast to direct policy optimization, which repeatedly differentiates the objective through the entire time-discretized dynamics, our method separates adjoint learning from policy improvement by explicitly incorporating state-sensitivity information from the Pontryagin stochastic maximum principle. For a fixed Markov control, we first compute pathwise adjoint labels by backpropagating the reward through time along simulated state trajectories, and project these labels onto the Markov state through a supervised regression step. The control is then updated by enforcing the pointwise Hamiltonian stationarity condition. We prove that the resulting projected adjoint process satisfies a discrete-time Pontryagin principle and converges, as the time step tends to zero, to the solution of the corresponding continuous-time adjoint backward stochastic differential equation. Numerical experiments illustrate the accuracy, stability, and scalability of the approach.

0.25

Reinforcement learning for constrained stochastic linear--quadratic control in continuous time

cs.LG math.OC ·2026-10-26 ·Weiping Wu, Qitong Pan, Xun Li, Peng Wu
similarity 0.50
Abstract

We study continuous-time stochastic linear--quadratic control with a scalar state and nonnegative control constraints. With quadratic running costs, randomized exploration can change the optimal mean feedback. We introduce an auxiliary problem without entropy in which the conditional policy mean is nonnegative and the covariance is prescribed in proportion to the squared state. Moment reduction and state separation yield two regional Riccati equations and characterize the finite-horizon value function and optimal conditional mean. We give a sufficient control--cost compatibility condition under which this mean coincides with the optimal feedback of the original constrained problem. For time-invariant coefficients, we characterize the undiscounted infinite-horizon optimum and establish mean square asymptotic stability under the stated assumptions. For unknown dynamics, we develop model-free actor--critic schemes for finite and infinite horizons. Numerical experiments compare the learned policies with their analytical targets and examine the effect of exploration on the optimal mean.

0.25

Addendum to "Optimal Steering of a Linear Stochastic System to a Final Probability Distribution, Part II"

eess.SY math.OC ·2026-10-29 ·Yongxin Chen, Tryphon T. Georgiou, Michele Pavon
theory similarity 0.50
Abstract · 4 pages

We hereby fill in a gap in an induction argument used in the proof of a theorem on state-covariance controllability of linear stochastic systems in an earlier paper by the authors. The mathematical statement is true. The missing point in the proof concerns the assertion that, in the canonical single-input case, the control can be assigned arbitrary endpoint values. We amend the argument by proving a slightly stronger induction statement that permits arbitrary finite endpoint jets of the control. An independent approach to homogeneous state-covariance controllability was provided in a paper by Abdelgalil and Georgiou (2025) as a corollary of strong controllability results on $\GLp(n,\mathbb R)$, and shown herein that it extends to the case of additive noise as well.

0.30

Minimum-Energy Control for Stochastic Linear Systems

eess.SY math.OC ·2026-10-09 ·Vaishnavi Sharma, Vaibhav Katewa
theory similarity 0.49
Abstract

The open-loop minimum-energy control (MEC) problem asks for control inputs that minimize the control energy while ensuring the terminal state reaches a desired value. In this paper, we study the open-loop MEC problem for stochastic linear time-invariant systems. We consider a variety of stochastic constraints that enforce the terminal error to be \emph{small}, and show that Stochastic MEC (SMEC) problem can be formulated as a QCQP. We obtain the optimal solution for SMEC and show that it generalizes the well-known Deterministic MEC problem by introducing novel stochastic variants of controllability and Gramian matrices. Finally, we show that there exists a fundamental lower bound on the optimal control energy which cannot be overcome by increasing the time horizon of the problem.

0.44