CS Paper Radar

Fixed-Time Resilient Integral Reinforcement Learning for Input-Constrained Unknown Nonlinear Systems Under FDI Attacks and Disturbances: A Data-Driven Admissible Warm Start

Tien Dat Vu, Minh Doan

eess.SY ·submitted 2026-09-12 17:32 UTC · v1
hot direction

Abstract

This paper develops a resilient learning controller for unknown nonlinear systems operating under actuator limits, false-data-injection attacks, and external disturbances. The key idea is to learn a saturated secure policy directly from finite trajectory data while guaranteeing that both the learning error and the closed-loop state converge to compact neighborhoods within a uniform fixed time independent of initial conditions. An integral formulation removes the unknown drift from the implementable learning law, while stored informative data sustain learning after online excitation fades. To mitigate the closed-loop sensitivity to arbitrary critic initialization, pre-deployment data, which may also be reused from the replay stack, are lifted through a finite-dimensional Koopman representation to construct a stabilizing initial policy, whose inverse saturated-policy map provides a data-driven critic-weight warm start. The resulting controller preserves input constraints by construction and guarantees practical fixed-time robustness under persistent attacks and disturbances. The proposed learning and initialization architecture is further verified through a two-link robot stabilization example, where the results demonstrate rapid state recovery, bounded critic learning, reliable actuator-constraint satisfaction, and improved closed-loop behavior under informed critic initialization.

Closest papers

Nearest neighbours by meaning across the whole index.

Predefined-Time Resilient Integral Reinforcement Learning for Input-Constrained Unknown Nonlinear Systems Under FDI Attacks and Disturbances: A Fully Data-Driven Approach

eess.SY ·2026-09-10 ·Tien Dat Vu, Minh Doan
hot direction similarity 0.80
Abstract

This paper investigates optimal control for nonlinear systems with unknown dynamics, input constraints, disturbances, and adversarial signals. The objective is to develop a learning-based control method that allows the designer to prescribe the desired convergence time in advance. An integral reinforcement-learning framework is proposed to avoid requiring exact knowledge of the system dynamics while ensuring that the control input always satisfies the actuator constraints. Current and recorded data are combined to train the critic without requiring persistent excitation. The learning gain is selected directly from the prescribed convergence deadline. Lyapunov analysis is then used to establish practical predefined-time convergence of the coupled state-critic system in the presence of disturbances and adversarial channels. The effectiveness of the proposed method is validated through the stabilization control of a two-link robot manipulator.

0.46

Predefined-Time Integral Reinforcement Learning for Saturated Unknown Nonlinear Multi-Agent Systems Under FDI Attacks and Disturbances

eess.SY ·2026-09-12 ·Tien Dat Vu, Minh Doan
similarity 0.78
Abstract

This paper addresses secure leader-follower formation of unknown nonlinear multi-agent systems under actuator constraints, external disturbances, and false-data-injection (FDI) attacks. The graph-coupled coordination-error dynamics are formulated as local zero-sum differential games, where a nonquadratic input utility yields saturation-compatible secure policies and actuator-channel FDI and disturbances act as adversarial inputs. To eliminate explicit dependence on the unknown nonlinear drift, an integral Bellman-Isaacs identity enables critic-only learning from finite trajectory data. A two-power state-cost structure and a deadline-parameterized critic update connect optimal learning with predefined-time stabilization. Unlike fixed-time methods whose settling-time bound is determined by preselected gains, the proposed framework assigns the overall deadline first and allocates it among data informativity, critic learning, the reinforcement window, and formation convergence. Practical predefined-time convergence of the critic and formation errors to bounded residual sets is established independently of initial conditions, while secure actuator constraints are satisfied by construction. Simulations validate the framework under FDI attacks, disturbances, input constraints, and different initial conditions.

0.21

Predefined-Time Integral Reinforcement Learning for Unknown Nonlinear Systems via Inverse-Optimal Design

eess.SY ·2026-10-08 ·Tien Dat Vu
similarity 0.70
Abstract

This paper develops a new predefined-time integral reinforcement learning framework for optimal control of unknown nonlinear systems. The unknown drift is first approximated by a radial basis function (RBF) neural network, together with a data-driven online identification law for updating the corresponding neural weights. After the identifier converges to a sufficiently small neighborhood of the true dynamics, the learned model is incorporated into the integral reinforcement learning (IRL) problem. Unlike conventional reinforcement-learning-based optimal control, the desired convergence time is introduced directly into the control objective: a Lyapunov function and its prescribed decay behavior are specified by the designer, and inverse-optimal control is then used to construct a compatible running cost whose optimal policy inherits the predefined-time stabilization property. The value function is approximated by a second RBF neural network, and a new critic update law is developed to impose predefined-time convergence on the critic weights. Finite informative learning data are stored in a replay buffer and reused during the critic update, thereby avoiding the need for persistent excitation throughout the closed-loop operation. Theoretical analysis proves that the critic-weight error enters a prescribed residual set within the allocated learning horizon, while the closed-loop state reaches a small neighborhood of the origin within the overall designer-specified deadline. Numerical simulations on an unknown nonlinear system verify accurate drift reconstruction, predefined-time critic learning, and closed-loop convergence.

0.37

Fixed-Time Integral Reinforcement Learning for Saturated Nonlinear Multi-Agent Systems Under FDI Attacks

eess.SY ·2026-09-05 ·Tien Dat Vu, Minh Doan
similarity 0.70
Abstract · 16 pages, 8 figures

The leader-follower formation control problem is investigated for nonlinear multi-agent systems with unknown dynamics, external disturbances, and false data injection (FDI) attacks on actuator channels. The problem is formulated as a zero-sum differential game and solved using the Integral Bellman-Isaacs approach. To address input saturation constraints, a non-quadratic control cost function is incorporated into the optimization problem, leading to a bounded control law. Furthermore, this paper proposes a cost function construction method and develops a critic learning law, which together guarantee the practical fixed-time stability of the system while overcoming the limitations of existing fixed-time reinforcement learning formulations. Finally, the practical fixed-time convergence of both the critic weight estimation error and the leader-referenced formation tracking error to bounded residual sets is rigorously proven. Simulation results demonstrate the effectiveness of the proposed method under external disturbances, FDI attacks, and input constraints.

0.50

Continuous-Time Policy Gradient from Discrete Data: Convergence Analysis and Model-Free Algorithms

cs.LG math.OC ·2026-10-20 ·Martin Hernandez, Xinyan Qing, Mo Zhou, Yuhua Zhu
similarity 0.69
Abstract

Continuous-time reinforcement learning seeks an optimal policy under unknown dynamics driven by stochastic differential equations using observations collected at discrete times. We develop a policy gradient framework that combines rigorous convergence analysis with learning from discrete data. We first establish a performance difference lemma and derive policy gradient formulas for randomized and parametrized policies. We then prove exponential convergence of an entropy-regularized policy flow to the regularized optimum. For discrete parametrized updates, we derive a convergence bound that explicitly separates the contributions of critic error, policy-regression error, and the learning step size. To enable learning under unknown dynamics, we integrate this framework with PhiBE, which constructs a continuous-time surrogate generator from discrete transition data. The resulting on-policy and off-policy model-free actor-critic algorithms preserve the continuous-time structure of the control problem. Numerical experiments on an entropy-regularized linear-quadratic regulator and a nonlinear wheeled robot demonstrate the performance and stability of the proposed methods with both known and data-driven dynamics.

0.12

Koopman Model-Based Reinforcement Learning

cs.RO eess.SY ·2026-11-10 ·Wenjian Hao, Paulo C. Heredia, Shaoshuai Mou
similarity 0.67
Abstract

We propose a Koopman model-based reinforcement learning (K-MBRL) algorithm for controlling nonlinear robotic systems with unknown dynamics. The K-MBRL learns a control policy by minimizing an approximate policy objective that combines the analytical immediate cost with an estimate of the future cost evaluated at the one-step predicted state. To this end, we develop a stochastic Koopman operator-based learning method to approximate the unknown nonlinear dynamics in a linear representation, while a critic function estimates the remaining future cost. This formulation offers two key advantages. First, the linear Koopman dynamics simplify the model-dependent policy gradient computation to a constant matrix product, enabling computationally efficient policy improvement. Second, one-step model propagation mitigates the accumulation of prediction errors associated with long-horizon rollouts while retaining long-term cost information through the critic. The learned policy enables real-time control through feedforward evaluation without online optimization. We establish policy learning convergence guarantees under standard stochastic approximation assumptions. Simulations on HalfCheetah and quadruped four-legged locomotion tasks demonstrate strong sample efficiency and improved performance compared with classical model-free and model-based reinforcement learning baselines. A pendulum swing-up task is used to assess policy gradient approximation accuracy and conduct ablation studies on the learning mechanism and key hyperparameters. The proposed K-MBRL is further validated on a physical quadruped robot performing a challenging sequence of maneuvers: rising to an upright posture supported by two rounded rear feet, maintaining balance during bipedal walking, and safely returning to the ground, as shown in Fig.1.

0.10

A Decade of Learning-Based Control: From Theoretic Foundations to Practical Deployment

eess.SY cs.LG ·2026-10-21 ·Angela P. Schoellig, Matthias A. Müller, Sebastian Trimpe, Melanie N. Zeilinger, Mahathi Anand +6 more
similarity 0.67
Abstract

Motivated by ten years of Learning-Based Control Invited Sessions at the IEEE Conference on Decision and Control (CDC), this tutorial paper reviews key advances at the intersection of machine learning and control. We focus on methods that combine the flexibility of machine learning techniques with the structure and theoretical foundations of control to design autonomous systems that are capable, reliable, and safe. We first introduce core notions for establishing stability and safety guarantees, then review approaches that learn dynamics models to improve control under uncertainty, and finally discuss data-driven methods that directly synthesize or learn policies, including reinforcement learning, imitation learning, and Bayesian optimization. We further discuss representative applications and open challenges. The goal is to provide an accessible entry point to the field and to support continued progress toward reliable learning-based control in real-world systems. To support learning and practical exploration, we share implementations and illustrative examples along with our discussion, together with additional resources on our website: https://learning-based-control.github.io/tutorial/.

0.51

Policy-Conditioned Koopman Representations for Parameterized Closed-Loop Systems

eess.SY ·2026-11-13 ·Kiran G. Krishnan, Shankar A. Deka
similarity 0.65
Abstract

Predicting the response of a nonlinear system across different feedback controllers requires a model that captures how the closed-loop dynamics change with the controller parameters. In a finite-dimensional Koopman model, these changes may affect both the latent dynamics and the observable coordinates used to represent them. We propose a policy-conditioned Koopman representation framework to address this problem. It allows a single learned model to predict a family of closed-loop trajectories without retraining for each controller. We test the framework on six nonlinear prediction benchmarks, where the mean error of the model stays below a 15% threshold on every system, while that of each baseline exceeds it on at least one. As an example application, the resulting surrogate supports one-shot feedback-parameter selection, in which an optimization at the initial state selects a controller that then operates with fixed parameters and ongoing state feedback. On nonlinear control benchmarks including MuJoCo, the selected controllers stabilize every system from the tested initial state with 1.4 to 21 times less online computation than nonlinear model predictive control, counting the selection itself. The results show that policy-dependent coordinates can support both prediction across closed-loop systems and controller selection without repeated online trajectory optimization.

0.11