CS Paper Radar

CGDT-CESN: Correlation-Guided and Dynamics-Trained Clustered Echo State Network for Multivariate Time-Series Prediction

Hariharan Srinivasan, Masoom Bhargava, Sayan Gupta

cs.LG nlin.AO ·submitted 2026-10-15 07:32 UTC · v1

Abstract

Accurate prediction of multivariate time series requires models that simultaneously capture variable-specific dynamics and cross-variable interactions. In this work, a Correlation-Guided and Dynamics-Trained Clustered Echo State Network (CGDT-CESN) is proposed, which comprises a modular reservoir architecture with each cluster distinctly specializing in one of the variables in the multivariate time series. An innovative reservoir design is proposed in which inter-cluster connections are enabled by empirically observed correlations among variables, providing a data-driven coupling mechanism. Apart from conventional readout training, inter-cluster connection weights informed by correlations are trained, which in turn allows the reservoir to capture time-varying cross-variable dependencies without sacrificing computational efficiency and further improves forecasting accuracy. The prediction performance of CGDT-CESN is found to be superior to that of the conventional ESN when evaluated on two fundamentally different datasets: Twitter information cascade evolution and solar wind forecasting. Also, the improvement in performance is preserved across multiple prediction horizons, even when presented with a low volume of data. By unifying cluster-specific specialization with learned inter-variable coupling, CGDT-CESN offers a principled, data-driven route to multivariate time series prediction.

Closest papers

Nearest neighbours by meaning across the whole index.

RC Prediction: A Python Package for Parameter-Aware Reservoir Computing

cs.LG eess.SYmath.DS ·2026-10-29 ·Jin-Cheng Rao, Zheng-Meng Zhai, Ying-Cheng Lai
benchmarkcode/data similarity 0.60
Abstract

A nonlinear dynamical system can be characterized by the autonomous evolution of its state in time and its dependence on system parameters. As a parameter varies, the dynamics can change quantitatively and, at bifurcation points, qualitatively. When the governing equations are unknown, a data-driven model should therefore capture both elements from observational data, rather than reproduce the dynamics at a single operating condition only. Parameter-aware reservoir computing (PARC) is a model-free framework for this purpose, where the parameter is supplied to a reservoir computer through an additional input channel. PARC is trained on time series recorded at several parameter values, and the trained reservoir, run in closed loop as an autonomous dynamical system, generates trajectories at parameter values that were never observed. rc_prediction is an open-source Python package that implements this workflow. It provides multi-parameter training of a reservoir, autonomous closed-loop prediction at user-specified parameter values, parameter scans, and a standard echo state network for forecasting at a fixed parameter. Hyperparameter optimization and optional GPU acceleration through PyTorch are also provided. We use three benchmark dynamical systems as examples to demonstrate its usage. As one of the featured applications, here we illustrate the prediction of critical transitions and system collapse: trained only on pre-critical data, the model is queried beyond the training range to predict whether the system collapses, to estimate the critical parameter $p_c^{*}$, and to characterize transient lifetimes.

0.19

Sparse Federated Echo State Networks for Time-Series Classification via Smooth-L1 Regularization

cs.LG ·2026-10-15 ·Gengcheng Lyu, Manuel Röder, Frank-Michael Schleif
publishedcode/data similarity 0.58
Abstract · 30 pages, 8 figures, 7 tables. Extended version of a paper presented at IDEAL 2025. Code: https://github.com/oroboro235/FedESN-SmoothL1. The…

Echo State Networks (ESNs), as lightweight implementations of reservoir computing, provide efficient solutions for temporal data processing in resource-constrained environments. While standard ESNs offer computational efficiency, their dense readout layers inflate the stored model and the per-sample inference cost. This paper introduces a Smooth-L1 regularized ESN (SL1ESN) that promotes sparsity in the readout layer while preserving differentiability for gradient-based optimization. The method employs a twice-differentiable Smooth-L1 approximation compatible with Newton's method, which we exploit both for centralized training and for a second-order (Newton-step) federated aggregation scheme. Under a single fixed shrinkage threshold shared by every dataset and both settings, experiments on seven time-series classification benchmarks show that SL1ESN learns readouts that are 77-95% sparse in the centralized setting and 75-95% sparse in the federated one. On five of the seven benchmarks, the mean accuracy of the sparse readout over ten seeds differs from that of the stronger dense baseline by at most 0.7 points in the centralized setting and 1.6 points in the federated one, differences of the same order as the seed-to-seed variation, while on the remaining two it trails by 2.4-5.8 points. The second-order federated update converges in only a few rounds. The resulting sparsity compresses the stored model, the per-sample inference cost, and the server-to-client broadcast by a comparable factor. These results demonstrate the practical value of sparse readouts for edge computing and IoT deployments.

0.44

Impact of Negative Weights on Performance in Reservoir Computing

cs.LG physics.soc-ph ·2026-11-04 ·Yeonsu Jeong, Seonho Nam, Dahan Choi, Mi Jin Lee, Seung-Woo Son
benchmark similarity 0.58
Abstract

Reservoir computing provides an efficient framework for time-series prediction because only the readout layer is trained while the recurrent reservoir remains fixed. In this study, we examine how the negative sign ratio of a signed Erdos-Renyi reservoir affects the internal dynamics and forecasting performance of an echo state network on the Mackey-Glass benchmark. To characterize the reservoir responses, we use two Shannon-entropy-based observables, namely node-state entropy and node-frequency entropy, as empirical proxies for the diversity of reservoir states and temporal activation patterns. We find that intermediate negative sign ratios are associated with richer internal dynamics and substantially lower prediction error than the sign-homogeneous extremes for the task considered here. These results suggest that link-sign composition can be a useful structural parameter in reservoir design for chaotic forecasting. The present findings should be interpreted within the scope of the current benchmark and reservoir setting, and broader validation will be required to establish their generality across other tasks, topologies, and controlled spectral conditions.

0.21

Washout-free synchronization, parameter estimation and uncertainty quantification of echo state networks

cs.LG nlin.CD ·2026-11-15 ·Andrea Nóvoa
code/data similarity 0.58
Abstract · 15 pages, 12 figures

Echo state networks (ESNs) forecast chaotic systems for several Lyapunov times. Like all recurrent networks, ESNs require a washout before forecasting, i.e., a phase in which the full state drives the network until the reservoir state synchronizes with the system. At deployment, however, sensors measure only part of the state, with noise and at a lower sampling rate than the training data. We propose a washout-free deployment of ESNs via ensemble data assimilation. In closed loop, the ESN is a Markovian dynamical system; hence, a sequential filter can correct its reservoir state whenever data are available. The filter synchronizes an ensemble of reservoir states, drawn from the training data, with the system, and quantifies the forecast uncertainty. We extend the method to parametric ESNs, to infer physical parameters, and to parallel ESNs with localization, to forecast high-dimensional systems. We test the method on the Lorenz-63 and Lorenz-96 systems. First, the filter synchronizes the ESN with a fraction of the data of a washout. Second, the filter deploys one full-state ESN from a subset of its components, which a washout cannot do. Third, the filter estimates a forcing that is not in the training data, and the estimated forcing lengthens the forecast. Fourth, localization synchronizes a parallel ESN with eight times fewer members than the global filter. Fifth, the ensemble spread signals, without the true state, when the ESN synchronizes and when the filter fails. This work enables the deployment of ESNs from partial, sparse and noisy observations.

0.35

Inference of Unknown Dynamical Components Using Next Generation Reservoir Computing: From Chaotic Systems to Climate Data

cs.LG nlin.CD ·2026-09-21 ·Jule Budnick, Andrew Keane, Serhiy Yanchuk
similarity 0.57
Abstract

We investigate next generation reservoir computing (NGRC) as a data-driven approach for inferring unseen components of dynamical systems. We compare NGRC with traditional reservoir computing (RC) using the Lorenz and Rössler system, where two unknown components are inferred from one given component. For both systems, NGRC achieves accurate results while requiring fewer training data and less computational time than RC. We identified an inverse proportional behavior between the number of time-delayed steps needed for NGRC and the temporal resolution, indicating that the physical time span covered by the delay interval is an important factor in determining the required number of delayed steps. Finally, we apply NGRC to the observational climate data of ENSO (El Niño--Southern Oscillation) and infer one observable from the remaining variables. Despite the noise and complexity of the real-world data, the NGRC shows promising results. Our findings demonstrate the potential of NGRC for efficient inference of unseen components in both controlled dynamical systems and real-world data.

0.32

Channel-Dependent State Space Model for Multivariate Time Series Forecasting

cs.LG cs.AI ·2026-09-29 ·Yu-Cheng Wu, Fan-Keng Sun, Li-Chun Lu, Duane S. Boning
benchmark similarity 0.57
Abstract

Multivariate time series forecasting (MTSF) is critical across many real-world domains. Existing deep learning approaches fall into two paradigms with distinct limitations: channel-independent (CI) methods unconditionally ignore cross-variable dependencies and model only temporal dynamics, while channel-dependent (CD) methods consider both but typically rely on architectural compromises to mitigate overfitting and computational overhead. We therefore propose Chameleon, a specialized CD state space model (SSM) that enables data-dependent, fine-grained interactions across variables while scaling linearly with their number. By connecting selective SSMs with the Kalman filter, we leverage the missing measurement update in the former for cross-variable modeling while preserving the SSM backbone for robust temporal modeling. We further identify favorable inductive biases of GatedDeltaNet for time series, adapt it as our backbone, and improve generalization through additional techniques, including a previously unexplored stochastic perturbation of reversible instance normalization. On strongly dependent ODE and PEMS datasets, Chameleon achieves the best MSE and MAE across all settings, while its CI ablation and prior CD methods incur 61-178% higher MSE on average. Across 28 standard benchmark settings, Chameleon also achieves better MSE and MAE than each baseline in at least 27 and 22 cases, respectively. Training-time and peak-memory analyses on Traffic and ETT further demonstrate competitive efficiency and favorable memory scalability across different variable counts.

0.29

Distributed JEPA: A Self-Supervised Framework for Energy Forecasting

cs.LG cs.AI ·2026-09-15 ·Liana Toderean, Tudor Cioara, Vasilis Michalakopoulos, Efstathios Sarantinopoulos, Ionut Anghel +1 more
hot direction similarity 0.56
Abstract

Traditional energy forecasting solutions rely on task-specific supervision and energy asset representations, limiting transferability and the ability to capture general temporal dynamics across heterogeneous assets. We address this by proposing a distributed Joint Embedding Predictive Architecture (JEPA) for self-supervised learning from heterogeneous energy time-series. The framework predicts latent representations of masked temporal segments while integrating temporal observations and contextual information within a shared embedding space. To prevent representation collapse, training combines a latent-space predictive objective with covariance and temporal variance regularization. The evaluation was conducted on energy consumption and generation datasets under data-degradation scenarios and compared with a Transformer forecasting baseline. The learned representations remained stable (cosine similarity $\approx 0.98$; effective rank 185-235). JEPA achieved performance comparable to a Transformer on building energy data, higher $R^2$ in 3/5 consumer clusters, and outperformed the baseline on 9/10 unseen PVs ($R^2$=0.73-0.88 vs. <0.45), while showing greater robustness to missing data.

0.28

Batched NN 4D-Var: a performant hybrid model framework for the Integrated Forecasting System

cs.LG stat.ML ·2026-10-23 ·Alban Farchi, Marcin Chrust, Massimo Bonavita, Marc Bocquet, Wei Pan +1 more
similarity 0.56
Abstract

Hybrid forecasting systems combining machine learning and numerical weather prediction offer a promising route toward improved forecast skill while preserving physical consistency. In this study, we investigate online learning of state-dependent model-error corrections within the Integrated Forecasting System using 4D-Var. Building on previous work, we represent model error using a neural network embedded within the forecast model and trained from observations through the data assimilation cycle. First, we introduce a targeted variant of neural-network 4D-Var in which only the neural-network parameters are estimated, while the atmospheric state is provided by a precomputed analysis. We then propose a batched extension that processes assimilation windows in random order and averages parameter increments over multiple windows, drawing inspiration from mini-batch stochastic optimization. Experiments are conducted with the Integrated Forecasting System of the European Centre for Medium-Range Weather Forecasts at an intermediate resolution. Sequential online learning substantially improves forecasts within the training period, with error reductions exceeding 10%in some cases. However, these improvements fail to generalize to unseen data. In contrast, the proposed batched formulation produces hybrid models that consistently improve forecast skill in out-of-sample experiments. Larger batch sizes accelerate convergence and improve robustness by reducing the noise of parameter updates. Diagnostics further show that the online-trained corrections preserve realistic forecast variability and do not improve skill through artificial smoothing. These results demonstrate that the formulation of the online learning problem is critical for generalisation and establish batched online training as a promising framework for hybrid machine-learning/data-assimilation systems.

0.26