CS Paper Radar

Turnpike properties in nonlinear system identification

Julian D. Schiller, Matthias A. Müller

eess.SY math.OC ·submitted 2026-09-02 03:55 UTC · v1

Abstract

We analyze the problem of learning general discrete-time nonlinear state-space models using the simulation error minimization (SEM) method. In this setting, model parameters are typically learned by minimizing the mismatch between simulated and measured outputs over a training dataset, or shorter subsequences extracted from it. Specifically, we study the cumulative output turnpike property of the underlying SEM optimization problem, which requires optimal output sequences emanating from a fixed initial state to approach and remain close to an optimal output sequence of the corresponding SEM problem with free initial state. In the presence of non-unique optimal output sequences---as may arise, for instance, in fully black-box system identification using neural networks---the property is formulated with respect to the closest such sequence. Turnpike behavior is generally desirable in practice, as it provides a theoretical justification for employing computationally more tractable SEM formulations with fixed initial states while ensuring that their optimal output sequences remain close to unconstrained optimal ones. Under a mild reachability condition, we establish equivalence between the cumulative turnpike property, coercivity of the value function, and a tailored notion of strict dissipativity. We additionally introduce a cardinality turnpike property and show that it is strictly weaker than the cumulative notion. Finally, we establish sufficient conditions for turnpike behavior based on incremental output stability, convexity of the stage cost, and a suitable optimality condition, and illustrate the theory by means of a numerical example.

Closest papers

Nearest neighbours by meaning across the whole index.

A Reinforcement Learning Approach to Space-Filling Input Design for Nonlinear System Identification

eess.SY ·2026-11-13 ·Charly Witmeur, Gonçalo G. Cruz, Maarten Schoukens, Mark C. Runacres, Jan Decuyper
benchmark similarity 0.64
Abstract

In this paper, we propose a reinforcement learning (RL) approach to space-filling input design for nonlinear system identification, in which an agent learns a control policy that actively drives the system to explore its state space. Classical input design, developed mainly for linear models, minimises the covariance of the estimated parameters. For nonlinear black-box models, however, the dominant error is the bias induced by the model structure over the region of interest, which motivates space-filling strategies that seek to cover the system state space. Existing space-filling approaches rely on predefined input signal classes and require solving explicit optimisation problems. Unlike these methods, the proposed approach requires no input class to be selected and no frequency band to be tuned for the system at hand, satisfies an amplitude bound by construction, and adapts to varying initial conditions through its observation-based policy without re-optimisation. The learned excitation signals exhibit physically meaningful behaviours, such as resonance excitation and amplitude modulation. The approach is evaluated against tuned swept sine and crest-factor-optimised multisine excitations, and against a state-of-the-art space-filling multisine design, on two nonlinear benchmark systems: a mass-spring-damper with nonlinear stiffness and a Van der Pol oscillator. Under the same amplitude bound, the learned policy achieves the best overall space-filling performance on both systems and distributes its samples more uniformly than the state-of-the-art design.

0.30

Adaptive Stability-Constrained Neural Differential Equations for Controlled Dynamical Systems with Unknown Inputs

eess.SY ·2026-08-10 ·Syed Pouladi
similarity 0.61
Abstract

Continuous-time neural models are attractive for identifying nonlinear systems, but a small one-step error can grow rapidly when a learned vector field is rolled out under inputs that differ from those used for training. This paper develops an adaptive stability-constrained neural differential equation (AS-NDE) for systems with measured controls and unmatched, unknown perturbations. The nominal vector field and a state--input-dependent Riemannian metric are learned jointly. Positive definiteness is enforced by construction, while a sampled differential inequality penalizes violations of a prescribed contraction rate. An incremental input-to-state bound is derived: the distance between two trajectories decays exponentially up to gains determined by differences in their controls and disturbances. The statement explicitly accounts for the time derivative of an input-dependent metric, a term that is easily omitted in heuristic stability regularizers. We give a reproducible evaluation protocol for a forced Duffing oscillator and a permanent-magnet synchronous motor (PMSM) model. Because no measured data or executed training runs accompany this draft, all numerical curves and tables are clearly identified as illustrative synthetic placeholders; their PGFPlots coordinates are embedded in the source for direct replacement. The resulting manuscript is intended as a technically consistent starting point, not as evidence of empirical superiority before the prescribed experiments are run.

0.04

Data-Driven Hamiltonian for Neural Optimal Control

eess.SY ·2026-10-26 ·Christopher A. Strong, Aswin Surya, Namhoon Cho, Jason J. Choi, Claire J. Tomlin
benchmark similarity 0.60
Abstract · 21 pages, 9 figures

Reinforcement learning (RL) has emerged as a prominent data-driven approach to optimal control, but often requires substantial amounts of data and provides limited guarantees when encountering out-of-distribution state-action pairs. In this paper, we propose an alternative approach grounded in the well-established Hamilton-Jacobi framework in optimal control theory. Rather than requiring an explicit analytical model of the system dynamics to evaluate the Hamiltonian, we replace this with a data-driven Hamiltonian (DDH), which we generalize to a broad class of optimal control problems. In contrast to standard RL approaches, the DDH provides a principled mechanism, requiring only minimal prior system information such as a Lipschitz constant, for quantifying and accounting for uncertainty in regions not covered by the data, enabling the derivation of controllers with performance guarantees. Additional prior system knowledge, when available, can be readily incorporated to improve performance. For high-dimensional systems, we further introduce a physics-informed neural approximation of the value function, trained using a novel procedure that generates worst-case fictitious rollouts evaluated through the DDH. On an LQR problem, the DDH reaches near-optimal cost on every dataset we test, including random data on which offline RL methods remain far from optimal, and needs far fewer samples than online RL. On a nonlinear lunar lander benchmark, the DDH achieves an optimality gap to a trajectory-optimization reference more than $3\times$ smaller than CQL and FQL, and matches the performance of SAC with approximately $37\times$ fewer samples. These results establish the DDH as a promising framework for data-driven optimal control, particularly in settings where available data are limited in quantity or quality.

0.33

Truncated backpropagation through time induces a two-sided gradient turnpike property

eess.SY cs.LGmath.OC ·2026-11-05 ·Julian D. Schiller, Matthias A. Müller
benchmark similarity 0.60
Abstract

Truncated backpropagation through time reduces the computational burden of training recurrent models by restricting gradient computations to shorter temporal contexts. This induces two distinct boundary effects: missing past information perturbs the forward state evolution, while truncating the backward recursion omits future information. This motivates decoupling the interval over which gradient contributions are averaged for the parameter update from the surrounding temporal context over which they are computed. Under exponential decay of the state-transition Jacobians and mild regularity conditions, this leads to a two-sided gradient turnpike property: the error between truncated- and full-context gradient contributions decays exponentially with the distance from both context boundaries. We derive bounds for the averaged update gradient, yielding guarantees on gradient alignment and approximate full-context stationarity; under unbiased interval sampling and a Polyak-Lojasiewicz condition, these further imply a bound on the resulting optimality gap. Numerical experiments on the Wiener-Hammerstein benchmark illustrate the role of past and future context and the trade-off between predictive performance and computational cost.

0.24

On Convergence of Gradient Temporal-Difference Learning with Regularized Corrections and Eligibility Traces

cs.LG math.OC ·2026-11-04 ·Jingyi Jin, Martha White, Hao Wang
similarity 0.59
Abstract · 66 pages, 1 figure. Submitted to the Journal of Machine Learning Research

Temporal-difference (TD) learning underlies many prediction and control algorithms, but can diverge when combined with off-policy learning, bootstrapping, and function approximation. TD with Regularized Corrections (TDRC) is theoretically sound and practically simple, but existing convergence theory covers only the one-step case under i.i.d. sampling, while the Huber Bellman-error formulation has not been extended to TDRC or eligibility traces. In this work, we establish convergence of constrained TDRC($λ$) with diminishing stepsizes and non-i.i.d. off-policy data, for state-dependent and history-dependent $λ$-parameters. We also propose TDRC($λ$)-Huber, which preserves the TDRC($λ$) fixed point while reducing sensitivity to large Bellman errors. For both algorithms, we apply an ODE-based weak-convergence method from stochastic approximation theory. For the nonlinear mean dynamics of TDRC($λ$)-Huber, we give an integral small-gain condition that guarantees uniqueness and global asymptotic stability of the target equilibrium. We then establish convergence of the constrained algorithm, derive computable sufficient conditions for convergence, and identify a regularization-based convergence mechanism shared by linear TDRC($λ$) and TDRC($λ$)-Huber. When the mean TD matrix is positive stable, both constrained algorithms converge under sufficiently strong regularization without requiring positive definiteness.

0.23

Neural Closed-Loop Identification of Nonlinear Systems with Stability and Generalization Guarantees

eess.SY cs.LG ·2026-10-20 ·Laura Meroi, Mahrokh G. Boroujeni, Leonardo Massai, Giancarlo Ferrari-Trecate
code/data similarity 0.59
Abstract · 16 pages, 11 figures. Preprint submitted to Automatica

Neural networks are expressive models of nonlinear dynamics, but identifying such systems from closed-loop data raises two difficulties: the feedback loop correlates inputs with measurement noise, biasing standard open-loop estimators, and the identified model must be stabilized by the data-collection controller, so that its closed-loop behavior matches the true plant's. We derive a complete parameterization of all nonlinear systems stabilized by a given controller, generalizing the classical dual-Youla parameterization to the nonlinear setting. Building on this characterization, we propose Internal Controller Identification (ICI), a computationally tractable framework for identifying potentially unstable systems while ensuring closed-loop stability by construction; in the linear case, our indirect method reduces to an open-loop identification problem, for which standard consistency results apply. The framework accommodates contextual information, letting a single certified model identify a family of systems, and supports a PAC-Bayesian training objective that, when data are scarce and prior knowledge is available, bounds the identified operator's expected prediction loss. Experiments show that ICI identifies open-loop unstable plants where uncertified baselines diverge, outperforms direct methods under weak excitation and feedback-correlated noise, generalizes zero-shot to environments unseen during training, and, with limited data, avoids the overfitting of empirical risk minimization while providing a non-vacuous generalization bound.

0.39

In-Context Estimation and Prediction for Parametric ODEs

cs.LG cs.AImath.NAmath.ST+1 ·2026-11-16 ·Abdullah Ahmed, Yun Yang
similarity 0.59
Abstract

Transformers have demonstrated the ability to perform in-context learning, that is, to adapt to new tasks from a few examples without updating their parameters. Existing theory has largely focused on emulating gradient-based algorithms for convex statistical problems or establishing general function approximation guarantees without exploiting the underlying statistical structure. We study in-context parameter estimation and trajectory prediction for parametric ordinary differential equations (ODEs), where estimation is generally nonconvex. Under suitable identifiability and regularity conditions, we construct a transformer that estimates unknown system parameters from noisy trajectory observations through a two-stage approximate maximum likelihood procedure: a coarse search provides an initialization within a locally strongly convex region, followed by gradient descent with local linear contraction. For prediction, we compose this estimator with a neural approximation of the ODE solution map. We further show that, through parameter sharing, a single transformer can perform both parameter estimation and trajectory prediction simultaneously. For these constructions, we establish nonasymptotic guarantees and finite-pretraining oracle inequalities that separate errors due to finite pretraining data, finite context length, and finite-sized transformer approximation. Numerical experiments on linear and nonlinear ODEs further show that pretrained transformers can benefit from longer inference contexts without updating their weights.

0.34

Dissipative Lagrangian Neural Networks for the Interpretable Modelling of Passive Mechanical Systems

cs.LG cs.CE ·2026-10-11 ·Rob Seabourne, Alice Cicirello
similarity 0.59
Abstract

In this paper, we propose the Dissipative Lagrangian Neural Network (dLNN), a continuous-time system identification technique that separately learns the conservative and dissipative dynamics of a mechanical system directly from data. A key contribution is a novel parametrisation of dissipative forces that (i) is structurally constrained to be dissipative, enforcing passivity of the learned model, (ii) cannot absorb conservative mechanics, and (iii) retains sufficient expressivity to represent a broad family of state-dependent damping and regularised friction forces. The model is validated on prediction tasks on two real-world mechanical systems: trajectory prediction of a chaotic double pendulum and inverse-dynamics modelling of a six-degree-of-freedom industrial robot. Across these tasks, the dLNN demonstrates improved predictive performance over existing Lagrangian architectures while remaining competitive with a black-box neural model. The imposed physical structure also improves generalisation to unseen states, with the benefits becoming particularly pronounced when training data is limited. Finally, we introduce a symbolic distillation procedure that extracts closed-form expressions from the learned physical components. Applied to the industrial robot, this produces interpretable symbolic models that retain high predictive accuracy while reducing the computational overhead associated with evaluating neural Lagrangian models.

0.08