CS Paper Radar

A KKL Observer Perspective on Reservoir Computing

Anastasia Bizyaeva, Fernando Castaños, Jaime A. Moreno

eess.SY cs.LGmath.DSmath.OC ·submitted 2026-10-03 07:10 UTC · v1
theory

Abstract

Reservoir computing (RC) is a machine learning technique for data-driven modeling of dynamics for forecasting and control, primarily studied in computer science and physics literature with promising applications in neural network learning, physical computing, and neuroscience. Why reservoirs learn and how to choose good reservoir architectures are considered important open questions. We show that the RC problem is mathematically an extension of a classic problem in systems and control theory, the Kazantzis-Kravaris-Luenberger (KKL) observer design problem. As a consequence, many of the questions considered open for RC stand to benefit from a large body of theory in the mature KKL literature, non-exhaustively including on questions of embedding, transverse stability, local and global uniqueness guarantees, and effective data-driven solution constructions. Elaborating on this connection, we show that the surprising forecasting ability of reservoirs is in fact a direct consequence of the well-known observer internal model principle, derive an upper bound on the prediction error over a fixed forecast horizon, and provide a partial explanation for why linear readout training in RC works reasonably well. This work illustrates how classical ideas from systems and control can provide strong theoretical backing and open new questions for modern machine learning methods.

Comment
8 pages

Closest papers

Nearest neighbours by meaning across the whole index.

The internal model principle in reservoir computing

eess.SY math.OC ·2026-10-06 ·Danyal Saqib, Alberto Padoan
similarity 0.78
Abstract · Submitted to the American Control Conference (ACC) 2027

The paper studies learning in reservoir computing systems through the lens of the internal model principle. The key idea is to model training signals as outputs of an exogenous signal generator, thus connecting reservoir computing to output regulation, observer theory, and frequency-domain interpolation. For linear signal generators, we characterize the steady-state response of a linear reservoir through invariant subspaces, establishing connections with Luenberger observers and classical frequency-domain interpolation. For nonlinear signal generators, we obtain analogous results using invariant manifolds, connecting synchronization to Kazantzis-Kravaris/Luenberger observers and readout training to nonlinear moment matching. Case studies illustrate how our perspective explains possible instability of trained predictors and informs reservoir design.

0.11

Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees

eess.SY ·2026-11-10 ·Hannah Berin-Costain, Jun Liu, Kirsten Morris
theory similarity 0.66
Abstract · 7 pages, 1 figure, ACC 2027 conference submission

Physics-informed neural network (PINN) Kazantzis--Kravaris--Luenberger (KKL) observers require an approximate solution to a KKL partial differential equation (PDE) and an approximate inverse of this solution. Either approximation can leave a nonzero estimation bias even without measurement noise. In this paper, we remove both errors from the implemented observer by construction. We build an encoder containing an explicitly invertible neural map. Its closed-form inverse provides the internal state reconstruction used to evaluate the learned KKL residual and correct the latent dynamics. This construction makes the synchronization manifold exactly invariant for every value of the network weights. Based on contraction of the residual-corrected observer dynamics, we prove an input-to-state stability (ISS) result for the implemented error dynamics with respect to measurement noise. A direct consequence is that the estimation error converges exponentially to zero in the noise-free setting. We provide two numerical examples, the Van der Pol oscillator and the chaotic Lorenz system, in which the residual-corrected observers achieve exact noise-free convergence and improved error responses compared with released PINN--KKL baselines.

0.35

Inference of Unknown Dynamical Components Using Next Generation Reservoir Computing: From Chaotic Systems to Climate Data

cs.LG nlin.CD ·2026-09-21 ·Jule Budnick, Andrew Keane, Serhiy Yanchuk
similarity 0.64
Abstract

We investigate next generation reservoir computing (NGRC) as a data-driven approach for inferring unseen components of dynamical systems. We compare NGRC with traditional reservoir computing (RC) using the Lorenz and Rössler system, where two unknown components are inferred from one given component. For both systems, NGRC achieves accurate results while requiring fewer training data and less computational time than RC. We identified an inverse proportional behavior between the number of time-delayed steps needed for NGRC and the temporal resolution, indicating that the physical time span covered by the delay interval is an important factor in determining the required number of delayed steps. Finally, we apply NGRC to the observational climate data of ENSO (El Niño--Southern Oscillation) and infer one observable from the remaining variables. Despite the noise and complexity of the real-world data, the NGRC shows promising results. Our findings demonstrate the potential of NGRC for efficient inference of unseen components in both controlled dynamical systems and real-world data.

0.32

Correct then Forecast: Observer State-Space Models for Time Series Forecasting

cs.LG stat.ML ·2026-09-27 ·Alexis-Raja Brachet, Guillaume Clavier--Frémond, Abdelhakim Ziani, Pierre-Yves Richard, Céline Hudelot
similarity 0.63
Abstract

Time series forecasting requires extrapolating the dynamics of an observed process beyond the last available measurement. Yet recurrent forecasting models typically treat observations as inputs that directly control their latent dynamics. It leads to a regime change when these observations become unavailable at prediction time. Following a state-estimation perspective, we introduce Observer State-Space Models (OSSMs), a class of recurrent models that interprets the observed input time series as measurements of an underlying autonomous dynamical system. OSSMs explicitly separate latent-state propagation from measurement assimilation: a single transition governs the dynamics across both context and forecasting intervals, while available observations correct the estimated state through an observer. This formulation naturally exposes classical control-theoretic properties, including observability and convergence of the state estimation error. We further show that conventional and recent SSMs can be recovered as particular instances of our OSSM framework, thereby providing a unified interpretation of their recurrent dynamics and revealing modeling inconsistencies. We perform experiments across several benchmarks showing that OSSM achieves substantial improvements while maintaining the same parameter count and training setup as the corresponding SSM baseline. These results support a simple principle for recurrent forecasting: observations should correct the estimated latent state, rather than control the dynamics used to propagate it.

0.30

Context-dependent time-series prediction via HyperReservoirs

cs.LG nlin.AOnlin.CD ·2026-09-28 ·Kohei Tsuchiyama, Takatomo Mihana, Ryoichi Horisaki, André Röhm
similarity 0.62
Abstract

Time series prediction is a common application of reservoir computing. When the training and testing time series data contains multiple dynamical regimes, because an underlying parameter is changing, or the data in fact consists of multiple distinct systems, simple application of the reservoir computing principle produces high prediction errors. Here, we propose a HyperReservoir as an extended model of reservoir computing especially designed for such cases. The HyperReservoir combines a main reservoir with a smaller context reservoir, where the latter modulates the output weights of the former. This structure resembles the hypernetworks from deep neural network literature. However, in contrast, HyperReservoirs retain the simple training via linear regression of standard reservoir computing. We compare the proposed architecture with a conventional ESN, in which context acts at the input, and a full-matrix Conceptor, in which context modulates the reservoir state space. We evaluate all three models on time-series prediction tasks based on Lorenz and Rössler systems, including for varying bifurcation parameters and time sampling scales. We find that the HyperReservoir achieves the lowest mean test error in all three tasks, and particularly outperforms conceptors on data that is sampled from the same attractor but at different time scales.

0.28

Extrapolating learned dynamics across bifurcations based on generalized synchronization

cs.LG nlin.CDmath.DS ·2026-10-25 ·Lingfeng Chen, Masanobu Inubushi
similarity 0.62
Abstract

Predicting dynamics across bifurcations by extrapolating trained models is crucial in diverse fields of physics. Generalized synchronization provides a framework for understanding dynamical reconstruction by neural networks, especially in the context of reservoir computing. Building on this framework, we derive a formula combining pretrained readouts to reconstruct unseen chaotic dynamics. Using classical Lagrange interpolation theory, we derive rigorous bounds on readout errors and conjugacy residuals. Numerical examples demonstrate accurate prediction of chaotic dynamics across bifurcations.

0.20

RC Prediction: A Python Package for Parameter-Aware Reservoir Computing

cs.LG eess.SYmath.DS ·2026-10-29 ·Jin-Cheng Rao, Zheng-Meng Zhai, Ying-Cheng Lai
benchmarkcode/data similarity 0.62
Abstract

A nonlinear dynamical system can be characterized by the autonomous evolution of its state in time and its dependence on system parameters. As a parameter varies, the dynamics can change quantitatively and, at bifurcation points, qualitatively. When the governing equations are unknown, a data-driven model should therefore capture both elements from observational data, rather than reproduce the dynamics at a single operating condition only. Parameter-aware reservoir computing (PARC) is a model-free framework for this purpose, where the parameter is supplied to a reservoir computer through an additional input channel. PARC is trained on time series recorded at several parameter values, and the trained reservoir, run in closed loop as an autonomous dynamical system, generates trajectories at parameter values that were never observed. rc_prediction is an open-source Python package that implements this workflow. It provides multi-parameter training of a reservoir, autonomous closed-loop prediction at user-specified parameter values, parameter scans, and a standard echo state network for forecasting at a fixed parameter. Hyperparameter optimization and optional GPU acceleration through PyTorch are also provided. We use three benchmark dynamical systems as examples to demonstrate its usage. As one of the featured applications, here we illustrate the prediction of critical transitions and system collapse: trained only on pre-critical data, the model is queried beyond the training range to predict whether the system collapses, to estimate the critical parameter $p_c^{*}$, and to characterize transient lifetimes.

0.19

Impact of Negative Weights on Performance in Reservoir Computing

cs.LG physics.soc-ph ·2026-11-04 ·Yeonsu Jeong, Seonho Nam, Dahan Choi, Mi Jin Lee, Seung-Woo Son
benchmark similarity 0.61
Abstract

Reservoir computing provides an efficient framework for time-series prediction because only the readout layer is trained while the recurrent reservoir remains fixed. In this study, we examine how the negative sign ratio of a signed Erdos-Renyi reservoir affects the internal dynamics and forecasting performance of an echo state network on the Mackey-Glass benchmark. To characterize the reservoir responses, we use two Shannon-entropy-based observables, namely node-state entropy and node-frequency entropy, as empirical proxies for the diversity of reservoir states and temporal activation patterns. We find that intermediate negative sign ratios are associated with richer internal dynamics and substantially lower prediction error than the sign-homogeneous extremes for the task considered here. These results suggest that link-sign composition can be a useful structural parameter in reservoir design for chaotic forecasting. The present findings should be interpreted within the scope of the current benchmark and reservoir setting, and broader validation will be required to establish their generality across other tasks, topologies, and controlled spectral conditions.

0.21