CS Paper Radar

Data-Driven LQR with Uncertainty Refinement and Domain Randomization---Part~I: Certified Uncertainty Refinement

Hamidreza Modares

eess.SY ·submitted 2026-10-24 16:49 UTC · v1
theory

Abstract

Learning controllers from trajectories corrupted by process and measurement noise is difficult when the system matrices and the noise covariances are unknown: noisy regressors bias least squares, and model uncertainty cannot be quantified without the noise covariance, which can only be estimated from residuals of the uncertain model. This paper, Part~I of a two-part study, resolves both obstacles with a finite-sample framework that certifies, and iteratively refines, uncertainty sets for the model, the noise in the collected data, and the noise covariance. A closed-loop parametrization based on delayed instrumental variables (IVs) removes the bias and assigns to every candidate model exactly one noise sequence that reproduces the data. Learning this noise breaks the circular dependence: the noise sequences give the covariance estimate directly, models whose noise violates the prior or the statistical certificates are discarded, and the surviving models yield a tighter covariance certificate. Several delayed instruments further tighten the model set and give a certified test of data consistency. We prove finite-sample containment of the true model, noise, and covariance, recursive nesting, and convergence at the IV rate. Part~II uses these certificates for domain-randomized linear quadratic regulator synthesis.

Closest papers

Nearest neighbours by meaning across the whole index.

Data-Driven Stochastic LQR Control with Probabilistic Guarantees and Uncertainty Refinement

eess.SY ·2026-11-03 ·Amir Modares, Themistoklis Charalambous, Hamidreza Modares
similarity 0.57
Abstract

This paper presents a probabilistic framework for the direct, data-driven synthesis of the stochastic linear- quadratic output-feedback control for linear systems sub- ject to process and measurement noise. Using delayed input-output measurements, we derive a lifted parameter- ization together with an equivalent lifted linear quadratic cost, enabling controller synthesis without explicitly identi- fying the lifted dynamics. Two complementary closed-loop formulations, based on trajectory and covariance represen- tations, are then developed, whose uncertainties depend only on the realized but unknown lifted noise. We char- acterize the relative performance of the two formulations in finite-data regimes. High-probability uncertainty bounds are developed for realized noises, leading to a convex optimization solution to the problem, whose feasibility cer- tifies probabilistic mean-square stability (p-MSS) and per- formance guarantees. To reduce conservatism, Gaussian zonotopes are used to refine uncertainty sets by elimi- nating noise realizations inconsistent with measured data, and a novel intersection rule enables the consistent fusion of data-driven and physics-informed information. These components are integrated into an adaptive framework that unifies indirect (identification-based) and direct (control- oriented) data-driven linear-quadratic output-feedback con- trol. Unlike existing direct data-driven LQR methods that verify stability a posteriori, our by-construction approach achieves robust stability alongside reduced conservatism and superior performance. Simulation results show that the proposed controller approaches the ideal optimal solution while outperforming existing data-driven methods under noisy measurements.

0.19

Robust Data-Driven Output-Feedback Reference Governor with Quantified Model Uncertainty

eess.SY ·2026-11-16 ·Tam W. Nguyen
benchmark similarity 0.57
Abstract · 45 pages, 4 figures, 2 tables

This paper constructs a data-driven reference governor (DDRG) from input--output data and supplied deterministic disturbance bounds. A data-consistent model set quantifies the uncertainty remaining in each prediction direction. With exact output measurements, the DDRG propagates output-history uncertainty while retaining exact applied-reference histories and converts the resulting tubes into constraint margins. Under a common contraction condition in a selected history norm, explicit terminal feasibility inequalities certify infinite-horizon continuation under a held reference and yield a closed invariant admissible set. A scalar update selects the largest certified step toward the requested reference, with recursive feasibility and constraint satisfaction from certified initialization. A separate bounded-measurement-noise extension derives the errors-in-variables model set and the additional physical-output margin through a measured-history certificate. On the coupled benchmark, longer nested records reduce one-step prediction disagreement at the probe regressor, and propagation accounts for most of the gap between the tube and true-model envelopes at the reported constraint row and prediction step.

0.25

Iterating Consistency Models: Stability, Error Bounds and Noise Schedules

cs.CV stat.MLcs.LG ·2026-10-02 ·Alessio Spagnoletti, Abdul-Lateef Haji-Ali, Andrés Almansa, Alain Oliviero Durmus, Eric Moulines +1 more
similarity 0.57
Abstract · 27 pages, 6 figures

Consistency models (CMs) have become a leading approach for generating high-quality samples in few steps. However, adding steps can improve or degrade sample quality in ways that are highly sensitive to the schedule and that existing theory does not fully explain. To provide accuracy guarantees and guide CM sampler design, we analyze multistep CM sampling as a composition of noising and approximate denoising operators. Under explicit, verifiable stability assumptions, we derive a non-asymptotic error bound that separates contraction of the initialization error from accumulation of approximation error. The bound assigns distinct roles to the schedule: large early noise levels drive contraction, while small late noise levels control the residual bias. As a corollary, we obtain explicit constants for strongly log-concave and semi-log-concave targets. We further establish a complementary guarantee whose assumptions, one-step accuracy and stability, can be estimated for a given trained model. Experiments show that the contraction and approximation profiles entering our bounds can be reliably measured and closely match the predicted functional forms. Together, these results provide a meaningful convergence theory for multi-step CMs and a practical route to sampler design.

0.31

Data-Driven LQR with Uncertainty Refinement and Domain Randomization---Part~II: Fisher-Majorized Certified Policy Iteration

eess.SY ·2026-10-24 ·Hamidreza Modares
theory similarity 0.56
Abstract

Building on the uncertainty description developed in Part~I, this paper introduces a Fisher-majorized domain-randomized generalized policy iteration framework for data-driven linear quadratic regulator (LQR) synthesis. At each iteration, partial Bellman evaluation over a fixed batch of uncertainty realizations produces a structured Sylvester policy-improvement step. A residual-weighted Fisher curvature controls the policy-induced variation in the stationary covariance and converts the Sylvester model into a certified upper bound on the cost change, thereby guaranteeing descent for every accepted step. A check-and-correct mechanism then certifies stability over the entire uncertainty support, rather than only over the sampled realizations. Consequently, every iterate is robustly stabilizing and monotonically decreases the fixed-batch objective. The Fisher truncation and stability correction become inactive after finitely many times. The objective gap then contracts linearly to a floor determined by the dispersion of the realization-wise Riccati targets and by measurement noise. In the absence of measurement noise, this floor vanishes when the realization-wise targets coincide.

0.30

Exact Noise Limits for Bounded Scalar Stabilization Experiments

eess.SY math.OC ·2026-09-30 ·Alexey Peregudin, Ngoc Tuan Dinh
similarity 0.56
Abstract · 6 pages, 1 figure. Submitted to the American Control Conference (ACC)

How much unknown process noise can a bounded experiment tolerate while still establishing that the plant can be stabilized? We answer this question for scalar discrete-time systems with bounded inputs, exact state measurements, and a disturbance-energy budget proportional to the experiment length. We distinguish individual stabilizability of every consistent model, stabilization by one common gain, and a common quadratic certificate. We determine their exact noise ceilings for every drift. The last two requirements coincide; individual stabilizability generally tolerates more noise. Bounded periodic inputs approach the ceilings, while adversarial disturbances prevent their attainment. We also determine the long-horizon limits. For unstable plants, a short experiment can establish individual stabilizability at noise levels where every sufficiently long experiment fails. The same obstruction yields upper bounds for multidimensional systems with real eigenvalues. Input design may use plant knowledge, but certification uses only the recorded data and noise bound. The results therefore provide fundamental robustness benchmarks for unknown-plant designs.

0.41

Robust Data-Driven Output Feedback for Systems With Unknown Input Delays

eess.SY ·2026-10-31 ·Ngoc Tuan Dinh, Ba Huy Nguyen, Alexey Peregudin, Egor Dogadin
similarity 0.55
Abstract · 6 pages, 1 figure. Submitted jointly to IEEE Control Systems Letters (L-CSS)

In a history-feedback realization, an input delay increases memory without increasing the number of unknown physical coefficients. We use this structure to design output-feedback controllers for discrete-time systems with one input, one output, and known undelayed order, using noisy input-output data. For a known delay, we derive a necessary and sufficient linear matrix inequality for common-quadratic stabilization of all models consistent with the data. Optimizing its equation-error bound gives a robustness margin for the measured record. For an unknown constant delay in a known interval, we characterize noiseless delay ambiguity and give a sufficient condition for one controller to stabilize all models for every delay consistent with the error bound. Under stated excitation and nondegeneracy assumptions, this procedure is feasible for all sufficiently small equation errors. Numerical examples compare robustness at equal controller memory and show that certified stabilization can succeed while distinct delay models remain compatible with the same data.

0.31

KL-Regularized Policy Iteration for Adaptive LQR: Gain Updates and Probing from One Policy Improvement

eess.SY math.OC ·2026-11-09 ·Ajinkya Bhole, Mohammad Mahmoudi Filabadi, Guillaume Crevecoeur, Tom Lefebvre
similarity 0.53
Abstract

Policy-gradient methods for adaptive linear quadratic control improve the feedback gain of an unknown plant online by gradient-type steps computed from closed-loop data. A recent framework of such methods, policy-gradient adaptive control (PGAC), takes one gradient-type step per sample, which its indirect form computes from an estimated model and its direct form directly from the data. Because data generated by a fixed linear feedback cannot identify the plant, PGAC adds a probing perturbation to the input and leaves its covariance as a design parameter. We study the adaptive LQR problem through a KL-regularized formulation, in which the probing is the randomness of a Gaussian policy kept close to a Gaussian reference policy, so that the cost of the probing is part of the objective. One policy improvement of this formulation gives a gain update together with a probing covariance. We show that, for suitable references centred at the current gain, this gain update is exactly PGAC's Gauss-Newton or natural-gradient update with any step size below a bound. We repeat this policy improvement, each time against a tempered copy of the current policy, and refer to this scheme as KL-regularized policy iteration (KLPI). On a known plant and with tempering, the cost of KLPI's gains converges linearly to the optimal cost, and the probing covariance keeps a floor proportional to the regularization weight. On an estimated plant, under conditions like PGAC's (small updates and sufficiently informative data), the closed loop stays stable, the cost of the gains converges linearly up to a bias, and the probing covariance again keeps this floor. In simulations at equal probing cost, the shape of KLPI's probing covariance, which PGAC can adopt, lowers the extra cost caused by suboptimal gains on nearly all tested plants, whereas KLPI's use of past iterations in its gain update changes this cost little.

0.29

A Bayesian Framework for Safe Optimal Control of Unknown Systems from Input-Output Data

eess.SY cs.LGmath.OC ·2026-11-02 ·Robert Lefringhausen, Annalena Daniels, Michael Fink, Sandra Hirche
code/data similarity 0.52
Abstract · 15 pages, 4 figures, 2 tables. Submitted to Automatica. Code available at https://github.com/TUM-ITR/PosteriorScenarioPlanning.jl

As control engineering is applied to increasingly complex systems, learning-based control, which uses empirical data to reduce the reliance on first-principles models, has emerged as a promising paradigm. However, such approaches must cope with nonlinear and stochastic dynamics, the absence of state measurements, and scarce data, while ensuring safe operation under constraints, which requires rigorous uncertainty quantification. Whereas existing methods treat these challenges in isolation, this article proposes a Bayesian framework that jointly addresses them. A prior in state-space form is updated from input-output data using particle Markov chain Monte Carlo methods, yielding a representation of joint uncertainty over dynamics and latent states. Building on this representation, a validation procedure is developed that provides probabilistic guarantees for constraint satisfaction under arbitrary causal control laws. For control synthesis, a formulation penalizing the conditional value-at-risk of the constraint function is proposed, together with an adaptive calibration mechanism that adjusts the performance-robustness trade-off to achieve a desired robustness level. The probability of successful certification is characterized theoretically. The framework is extended to a receding-horizon setting, where posterior updates from new measurements allow the controller to progressively reduce uncertainty and improve performance. The approach is demonstrated on the optimal control of a vertical farm.

0.42