CS Paper Radar

Predefined-Time Resilient Integral Reinforcement Learning for Input-Constrained Unknown Nonlinear Systems Under FDI Attacks and Disturbances: A Fully Data-Driven Approach

Tien Dat Vu, Minh Doan

eess.SY ·submitted 2026-09-10 17:08 UTC · v1
hot direction

Abstract

This paper investigates optimal control for nonlinear systems with unknown dynamics, input constraints, disturbances, and adversarial signals. The objective is to develop a learning-based control method that allows the designer to prescribe the desired convergence time in advance. An integral reinforcement-learning framework is proposed to avoid requiring exact knowledge of the system dynamics while ensuring that the control input always satisfies the actuator constraints. Current and recorded data are combined to train the critic without requiring persistent excitation. The learning gain is selected directly from the prescribed convergence deadline. Lyapunov analysis is then used to establish practical predefined-time convergence of the coupled state-critic system in the presence of disturbances and adversarial channels. The effectiveness of the proposed method is validated through the stabilization control of a two-link robot manipulator.

Closest papers

Nearest neighbours by meaning across the whole index.

Fixed-Time Resilient Integral Reinforcement Learning for Input-Constrained Unknown Nonlinear Systems Under FDI Attacks and Disturbances: A Data-Driven Admissible Warm Start

eess.SY ·2026-09-12 ·Tien Dat Vu, Minh Doan
hot direction similarity 0.80
Abstract

This paper develops a resilient learning controller for unknown nonlinear systems operating under actuator limits, false-data-injection attacks, and external disturbances. The key idea is to learn a saturated secure policy directly from finite trajectory data while guaranteeing that both the learning error and the closed-loop state converge to compact neighborhoods within a uniform fixed time independent of initial conditions. An integral formulation removes the unknown drift from the implementable learning law, while stored informative data sustain learning after online excitation fades. To mitigate the closed-loop sensitivity to arbitrary critic initialization, pre-deployment data, which may also be reused from the replay stack, are lifted through a finite-dimensional Koopman representation to construct a stabilizing initial policy, whose inverse saturated-policy map provides a data-driven critic-weight warm start. The resulting controller preserves input constraints by construction and guarantees practical fixed-time robustness under persistent attacks and disturbances. The proposed learning and initialization architecture is further verified through a two-link robot stabilization example, where the results demonstrate rapid state recovery, bounded critic learning, reliable actuator-constraint satisfaction, and improved closed-loop behavior under informed critic initialization.

0.44

Predefined-Time Integral Reinforcement Learning for Saturated Unknown Nonlinear Multi-Agent Systems Under FDI Attacks and Disturbances

eess.SY ·2026-09-12 ·Tien Dat Vu, Minh Doan
similarity 0.77
Abstract

This paper addresses secure leader-follower formation of unknown nonlinear multi-agent systems under actuator constraints, external disturbances, and false-data-injection (FDI) attacks. The graph-coupled coordination-error dynamics are formulated as local zero-sum differential games, where a nonquadratic input utility yields saturation-compatible secure policies and actuator-channel FDI and disturbances act as adversarial inputs. To eliminate explicit dependence on the unknown nonlinear drift, an integral Bellman-Isaacs identity enables critic-only learning from finite trajectory data. A two-power state-cost structure and a deadline-parameterized critic update connect optimal learning with predefined-time stabilization. Unlike fixed-time methods whose settling-time bound is determined by preselected gains, the proposed framework assigns the overall deadline first and allocates it among data informativity, critic learning, the reinforcement window, and formation convergence. Practical predefined-time convergence of the critic and formation errors to bounded residual sets is established independently of initial conditions, while secure actuator constraints are satisfied by construction. Simulations validate the framework under FDI attacks, disturbances, input constraints, and different initial conditions.

0.21

Fixed-Time Integral Reinforcement Learning for Saturated Nonlinear Multi-Agent Systems Under FDI Attacks

eess.SY ·2026-09-05 ·Tien Dat Vu, Minh Doan
similarity 0.71
Abstract · 16 pages, 8 figures

The leader-follower formation control problem is investigated for nonlinear multi-agent systems with unknown dynamics, external disturbances, and false data injection (FDI) attacks on actuator channels. The problem is formulated as a zero-sum differential game and solved using the Integral Bellman-Isaacs approach. To address input saturation constraints, a non-quadratic control cost function is incorporated into the optimization problem, leading to a bounded control law. Furthermore, this paper proposes a cost function construction method and develops a critic learning law, which together guarantee the practical fixed-time stability of the system while overcoming the limitations of existing fixed-time reinforcement learning formulations. Finally, the practical fixed-time convergence of both the critic weight estimation error and the leader-referenced formation tracking error to bounded residual sets is rigorously proven. Simulation results demonstrate the effectiveness of the proposed method under external disturbances, FDI attacks, and input constraints.

0.51

Predefined-Time Integral Reinforcement Learning for Unknown Nonlinear Systems via Inverse-Optimal Design

eess.SY ·2026-10-08 ·Tien Dat Vu
similarity 0.68
Abstract

This paper develops a new predefined-time integral reinforcement learning framework for optimal control of unknown nonlinear systems. The unknown drift is first approximated by a radial basis function (RBF) neural network, together with a data-driven online identification law for updating the corresponding neural weights. After the identifier converges to a sufficiently small neighborhood of the true dynamics, the learned model is incorporated into the integral reinforcement learning (IRL) problem. Unlike conventional reinforcement-learning-based optimal control, the desired convergence time is introduced directly into the control objective: a Lyapunov function and its prescribed decay behavior are specified by the designer, and inverse-optimal control is then used to construct a compatible running cost whose optimal policy inherits the predefined-time stabilization property. The value function is approximated by a second RBF neural network, and a new critic update law is developed to impose predefined-time convergence on the critic weights. Finite informative learning data are stored in a replay buffer and reused during the critic update, thereby avoiding the need for persistent excitation throughout the closed-loop operation. Theoretical analysis proves that the critic-weight error enters a prescribed residual set within the allocated learning horizon, while the closed-loop state reaches a small neighborhood of the origin within the overall designer-specified deadline. Numerical simulations on an unknown nonlinear system verify accurate drift reconstruction, predefined-time critic learning, and closed-loop convergence.

0.37

Learning neural controllers for nonlinear systems from data

eess.SY ·2026-08-29 ·Zhongjie Hu, Zhi-Wei Liu, Chen Wang
similarity 0.67
Abstract

This article addresses the problem of designing neural feedback controllers for unknown nonlinear systems. We propose an indirect data-driven framework that uses offline data to identify the system dynamics, upon which a neural feedback controller and a neural Lyapunov function are jointly synthesized. Input constraints are enforced by integrating a hard-saturation structure into the controller architecture. Robust synthesis conditions are derived to account for data perturbations during identification. Formal stability is certified by combining SMT verification with local Lyapunov analysis near the equilibrium. Numerical examples validate the effectiveness of the proposed framework.

0.22

Inverse-Optimal Fixed-Time Integral Reinforcement Learning for Unknown Nonlinear Leader-Follower Multi-Agent Systems

eess.SY ·2026-11-11 ·Tien Dat Vu, Minh Doan
similarity 0.62
Abstract

This paper develops a distributed inverse-optimal fixed-time integral reinforcement learning framework for unknown nonlinear leader-follower multi-agent systems. The proposed method first reconstructs the unknown agent dynamics from finite integral data. Based on the identified dynamics, a distributed inverse-optimal formulation is constructed by prescribing a desired fixed-time value function and recovering a compatible running cost. A critic-only integral reinforcement learning architecture is then developed using current integral data and finitely informative experience-replay data, thereby avoiding the persistent-excitation requirement. Practical fixed-time convergence of the critic-weight errors is established in the presence of identification and function-approximation residuals. Finally, a network-level composite Lyapunov analysis proves practical fixed-time leader-following consensus of the true unknown nonlinear multi-agent system. Numerical simulations are provided to validate the developed theory.

0.18

Event-Triggered Practical Fixed-Time Integral Reinforcement Learning for Unknown Nonlinear Systems

eess.SY ·2026-09-30 ·Tien Dat Vu, My Nguyen Bach, Minh Doan
similarity 0.61
Abstract

This paper develops an event-triggered fixed-time integral reinforcement learning framework for optimal control of unknown nonlinear systems. An integral data-driven identifier is first used to reconstruct the unknown dynamics, after which an inverse-optimal formulation is employed to construct a fixed-time running cost. A learning law satisfying the practical fixed-time property is then derived. Previously collected data, or data obtained during a finite excitation interval, are stored in an experience-replay buffer and incorporated into the weight-update law. This avoids the persistent-excitation condition, which is often difficult to satisfy in practical operation. To reduce communication and control updates, an event-triggered mechanism is introduced. The paper shows that, under the event-triggered implementation, the closed-loop system still achieves practical fixed-time stability, while the proposed triggering rule guarantees the exclusion of Zeno behavior. Finally, a nonlinear example is presented to verify the theoretical results developed in the paper.

0.21

Stability-aware Residual Reinforcement Learning Framework for Robotic Manipulator Disturbance Compensation

cs.RO ·2026-09-18 ·Jihong Kim, Joonhyuk Kwon, Hwa Soo Kim, TaeWon Seo, Hyung-Tae Seo
similarity 0.60
Abstract · 14 pages, 9 figures

Although conventional controllers and disturbance observers (DOBs) are the standard for precision tracking in manipulators, they suffer from parameter uncertainty, nonlinear friction, and compound disturbances. This study proposes a residual reinforcement learning DOB framework that pairs an analytical observer with an RL policy. The deterministic baseline operates within a reliable region, whereas the RL policy explicitly targets the residuals that the model cannot capture. To make this compensation disturbance-aware, an estimator network aligns the observation history with a privileged disturbance context, organizing the latent space by disturbance regime and enabling rapid adaptation across disturbance transitions. To guarantee stability, we derived and enforced a state-dependent action bound on the RL policy from an input-to-state stability (ISS) analysis such that the closed loop provably confines the tracking error to a certified envelope for arbitrary policy outputs. Experiments on a 6-DOF manipulator demonstrated consistent improvements in disturbance estimation and tracking, including a 27.8% tracking-error reduction on real hardware under zero-shot sim-to-real transfer and a 38.0% reduction under a base-vibration disturbance that was not observed during training.

0.45