CS Paper Radar

Reference Replay for Excitation Injection and Subsequent Exponential Convergence under Concurrent Learning Adaptive Control

Tom Kaufmann, Johann Reger

eess.SY math.OC ·submitted 2026-11-16 16:09 UTC · v1
theory

Abstract

Concurrent learning (CL) guarantees precise identification of unknown parameters---if data with finite excitation (FE) happens to be collected online. To improve the reliability of CL, this work establishes a mechanism allowing FE-occurrence to be verified a priori: We propose replaying a given training segment in the reference until data with finite excitation is collected online. Provided the training segment fulfills a certain off-line excitation test, FE then occurs in unknown, but finite time. Moreover, we provide an upper bound on the time-to-FE. Even though this bound involves a dependency on the unknown initial Lyapunov-function value, its structure shows that FE occurs in arbitrarily early replay cycles when increasing the control gain; which may motivate employing a high-gain strategy. The theoretical findings are verified on an academic simulation example.

Closest papers

Nearest neighbours by meaning across the whole index.

Fixed-Time Resilient Integral Reinforcement Learning for Input-Constrained Unknown Nonlinear Systems Under FDI Attacks and Disturbances: A Data-Driven Admissible Warm Start

eess.SY ·2026-09-12 ·Tien Dat Vu, Minh Doan
hot direction similarity 0.64
Abstract

This paper develops a resilient learning controller for unknown nonlinear systems operating under actuator limits, false-data-injection attacks, and external disturbances. The key idea is to learn a saturated secure policy directly from finite trajectory data while guaranteeing that both the learning error and the closed-loop state converge to compact neighborhoods within a uniform fixed time independent of initial conditions. An integral formulation removes the unknown drift from the implementable learning law, while stored informative data sustain learning after online excitation fades. To mitigate the closed-loop sensitivity to arbitrary critic initialization, pre-deployment data, which may also be reused from the replay stack, are lifted through a finite-dimensional Koopman representation to construct a stabilizing initial policy, whose inverse saturated-policy map provides a data-driven critic-weight warm start. The resulting controller preserves input constraints by construction and guarantees practical fixed-time robustness under persistent attacks and disturbances. The proposed learning and initialization architecture is further verified through a two-link robot stabilization example, where the results demonstrate rapid state recovery, bounded critic learning, reliable actuator-constraint satisfaction, and improved closed-loop behavior under informed critic initialization.

0.44

On sparsity and directional forgetting in adaptive control

eess.SY ·2026-09-20 ·Tochukwu E. Ogri, Trivikram Satharasi, Muzaffar Qureshi, Kyle Volle, Rushikesh Kamalapurkar
similarity 0.64
Abstract

This paper develops a sparsity-promoting memory regressor extension (MRE) adaptation law with directional forgetting for nonlinear control-affine systems with linearly parameterized uncertainty. The objective is to use directional forgetting to selectively discount obsolete information and leverage $\ell_1$ regularization to promote sparsity of the parameter estimates. While $\ell_1$ regularization has been applied to the system identification problem in an offline setting, a contribution of this paper is to develop a recursive least squares update law to implement $\ell_1$ regularization in online adaptive control. In particular, we show that $\ell_1$-regularized recursive least squares is realized via a sliding mode update law. A nonsmooth Lyapunov-based stability analysis is then used to show that the tracking and parameter estimation errors are ultimately bounded under a subspace excitation condition. Simulation results on a Van der Pol oscillator demonstrate the ability of the developed sparsity-promoting MRE controller to recover sparse dynamics while maintaining stable tracking.

0.09

Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning

cs.LG stat.ML ·2026-08-12 ·Tieliang Gong, Zhongbo Zhang, Wen Wen, Yong-Jin Liu
benchmark similarity 0.64
Abstract

Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting. Yet its generalization behavior is shaped by two coupled effects that existing analyses fold into a single hypothesis-level quantity: finite memory replaces each past distribution with an empirical proxy, and repeated reuse couples the buffer, the current data, and the final hypothesis through a shared optimization trajectory. We develop a layer-wise information-theoretic framework that separates these effects at every depth. Our main result decomposes the expected generalization gap into a replay-induced representation drift and an optimization-dependence term, the latter further resolved into stability, plasticity, interaction, and residual-coupling components. Two refinements make the framework operational. A Wasserstein relaxation of the drift term, valid under support mismatch, yields a depth-dependent drift--sensitivity trade-off whose minimizer identifies which interior layer to stabilize. An SGLD instantiation of the optimization term reduces it to a trajectory-level log-determinant budget, exposing a curvature-aware gradient-alignment statistic that serves as an online diagnostic of task-wise forgetting. Controlled and benchmark experiments confirm the predicted memory scaling, the interior funnel, and the alignment signal's link to forgetting.

0.04

Closing the loop in learning with missing data

eess.SY cs.LGmath.OC ·2026-08-10 ·Dimitrios Pylorof, Humberto E. Garcia
similarity 0.64
Abstract

What should a machine learning model learn when data is missing during training? We look at the learning process from a dynamical systems perspective, cast data missingness as a structured loss of actuation that limits controllability of the parameter error dynamics, and ultimately derive adaptation mechanisms with Lyapunov stability characteristics that throttle model updates in ways that preserve learning coherence under partial, intermittent observability. Under recurrent excitation, our analysis provides ISS-type residual-to-state bounds with respect to a bounded closed-loop mismatch between the loss residual and the preconditioned update geometry. We evaluate the efficacy of our directional observability-aware adaptive learning approach on multimodal contexts, reinforcing its premise in promoting learning coherence and stability even in pathologically sparse domains and problems.

0.04

Entropic Control for Homeostatic Optimization: Pre-empting Catastrophic Forgetting via Dynamic Feedback

cs.LG cs.AIcs.CL ·2026-10-25 ·Shangxi Wu, Xiaorong Yan, Xiaojun Meng, Yuxiang Zhang, Jiansheng Wei
similarity 0.63
Abstract

Supervised Fine-Tuning (SFT) of LLMs suffers from catastrophic forgetting, and prevailing mitigation strategies struggle to reconcile the plasticity-stability dilemma. To address this, we propose Entropic Control for Homeostatic Optimization (ECHO), a closed-loop control framework. By decomposing the KL divergence, we identify output entropy as a dominant observable component of KL variation. This enables us to leverage entropy as a computationally efficient online proxy to monitor forgetting risks in real-time, bypassing expensive reference-model inference. Consequently, ECHO employs an elastic controller that dynamically modulates the gradient weight of replay data via a proportional control law, using deviations from the pre-trained entropic equilibrium as an error signal. Empirical evaluations across 0.6B to 9B parameter architectures demonstrate that ECHO lies on, and in several comparisons extends, the observed plasticity-stability frontier and maintains high operational robustness against varying data mixing ratios.

0.21

KL-Regularized Policy Iteration for Adaptive LQR: Gain Updates and Probing from One Policy Improvement

eess.SY math.OC ·2026-11-09 ·Ajinkya Bhole, Mohammad Mahmoudi Filabadi, Guillaume Crevecoeur, Tom Lefebvre
similarity 0.62
Abstract

Policy-gradient methods for adaptive linear quadratic control improve the feedback gain of an unknown plant online by gradient-type steps computed from closed-loop data. A recent framework of such methods, policy-gradient adaptive control (PGAC), takes one gradient-type step per sample, which its indirect form computes from an estimated model and its direct form directly from the data. Because data generated by a fixed linear feedback cannot identify the plant, PGAC adds a probing perturbation to the input and leaves its covariance as a design parameter. We study the adaptive LQR problem through a KL-regularized formulation, in which the probing is the randomness of a Gaussian policy kept close to a Gaussian reference policy, so that the cost of the probing is part of the objective. One policy improvement of this formulation gives a gain update together with a probing covariance. We show that, for suitable references centred at the current gain, this gain update is exactly PGAC's Gauss-Newton or natural-gradient update with any step size below a bound. We repeat this policy improvement, each time against a tempered copy of the current policy, and refer to this scheme as KL-regularized policy iteration (KLPI). On a known plant and with tempering, the cost of KLPI's gains converges linearly to the optimal cost, and the probing covariance keeps a floor proportional to the regularization weight. On an estimated plant, under conditions like PGAC's (small updates and sufficiently informative data), the closed loop stays stable, the cost of the gains converges linearly up to a bias, and the probing covariance again keeps this floor. In simulations at equal probing cost, the shape of KLPI's probing covariance, which PGAC can adopt, lowers the extra cost caused by suboptimal gains on nearly all tested plants, whereas KLPI's use of past iterations in its gain update changes this cost little.

0.29

Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem

cs.LG ·2026-08-21 ·Nazreen Shah, Naveen Kumar Reddy Somireddy, Zubair Shaban, Ranjitha Prasad, B. N. Bharath
published similarity 0.61
Abstract · Accepted at 5th Conference on Lifelong Learning Agents, 2026

Federated Continual Learning (FCL) is fundamental to real-world distributed learning systems, requiring models to adapt to sequential, non-IID data across clients while mitigating catastrophic forgetting and client drift. Existing approaches formulate continual learning (CL) as a sequence of per-task optimization problems, applied locally at each client and coupled through aggregation, using heuristic mechanisms such as replay, regularization, or projection-based constraints. However, forgetting in FCL is inherently a long-term, distributed phenomenon, arising from the interaction of temporal task evolution and cross-client heterogeneity, which is not explicitly regulated. In this work, we cast FCL as a stochastic control problem and propose Federated Queue-regulated Continual Learning (FedQCL), a framework based on Lyapunov drift-plus-penalty (DPP) optimization. FedQCL introduces virtual queues to track the accumulation of forgetting across tasks and clients, enabling explicit control of the stability-plasticity trade-off. By optimizing a DPP objective, the method jointly improves current-task performance while the queue-based formulation provides an interpretable and tunable mechanism to balance adaptation and retention through a single parameter, without requiring gradient projection or additional communication overhead. Empirical evaluations on standard benchmarks, including Split-CIFAR-10, Split-CIFAR-100, and Split-TinyImageNet, demonstrate that FedQCL outperforms state-of-the-art baselines with respect to accuracy while significantly reducing forgetting under heterogeneous data distributions.

0.21

Robust Model Reference Adaptive Control with Combined Adaptation under Finite Excitation Condition

eess.SY ·2026-08-23 ·Manish Patel, Arnab Maity
similarity 0.60
Abstract

In adaptive control, parametric uncertainties in linear-in-parameter form consist of unknown parameters and known regressor signals. Convergence of the unknown parameters to their ideal values requires the regressor to satisfy a persistent excitation (PE) condition, which depends on future data and is therefore infeasible to guarantee online. Memory-based parameter update laws address this by enabling ideal parameter convergence under the online-verifiable finite excitation (FE) condition. In this paper, a new algorithm is proposed to construct a memory term via the Modified Gram-Schmidt orthogonalization procedure for a class of multi-input multi-output nonlinear systems with an unknown diagonal control effectiveness matrix and bounded nonparametric uncertainties. Under the finite excitation condition, the constructed memory term yields an identity coefficient matrix in the parameter estimation error dynamics. The identity coefficient matrix eliminates the need for time-varying adaptation gains, enables an explicit ultimate bound on the parameter estimation error, and preserves the structure of the nonparametric uncertainty bound under the memory term. Building on this, a combined adaptation law is developed for controller gain estimation under FE. The closed-loop tracking and estimation errors are shown to decay exponentially to a neighborhood of the origin, characterized by an explicit ultimate bound, with a decay rate that depends solely on user-defined gains and system constants, independent of the level of regressor excitation. This removes the dependence of the convergence rate on the level of regressor excitation, a key limitation of existing approaches such as concurrent learning, memory regressor extension, and DREM.

0.24