CS Paper Radar

Why Directly Learning Periodic Trajectories Can Fail

Kaixin Zheng, Anita Layton

cs.AI math.CAmath.NA ·submitted 2026-09-26 05:19 UTC · v1
theory

Abstract

Operator learning of periodic solutions requires deciding how simulation data should be recorded and represented. A natural choice is to integrate long enough for transients to decay and record a window wide enough to contain at least one full period of all trajectories. We find that these conservative choices can make the resulting trajectories difficult to learn, even when the underlying periodic orbits vary regularly with system parameters. Unaligned trajectories generalize poorly even within the training distribution. Phase alignment substantially improves in-distribution generalization, but models trained on a fixed physical-time window still have large errors on trajectories with periods outside the training range. We explain both failures through a common mechanism: frequency differences accumulate over time, so the target phase varies rapidly with the parameters. Predictors that cannot track this variation incur a population MSE floor in both settings; for fixed window prediction, we also derive a per-sample lower bound. We then study one of the simplest representations that escape these floors: learning an aligned, normalized waveform and its period separately. We establish regularity of the decoupled targets under ODE assumptions and show experimentally that this approach avoids both failures in ODE systems and a PDE case study.

Closest papers

Nearest neighbours by meaning across the whole index.

Variational Trajectory Flow Matching with a Non-Linear Interpolant

cs.LO ·2026-11-19 ·Gurjeet Sangra Singh, Frantzeska Lavda, Alexandros Kalousis
similarity 0.62
Abstract

Simulation-free methods such as Trajectory Flow Matching learn dynamics by regressing the time derivative of a local interpolant placed between consecutive observations. The interpolant defines the regression target, and under sparse, irregular sampling, it underrepresents the dynamics and bounds what can be learned. We show that no fixed interpolant dominates: the best of linear, quadratic, spline and quintic changes with the system and with the sampling density. We propose to learn the interpolant instead. An amortized implicit neural representation (INR) is fitted to the training trajectories and frozen as a teacher supplying the target velocity to a learnable velocity field. We further frame the velocity field within a variational framework, yielding a probabilistic forecaster that, at inference, requires only the velocity field and no SDE solver. We evaluate our method on a range of dynamical systems: it consistently achieves high accuracy and more stable long-horizon rollouts under sparser observations, at matched velocity-field capacity.

0.27

Symbolic Neural ODEs: Learning interpretable models from time-series data

cs.LG eess.SYmath.DSstat.ML ·2026-08-22 ·Nibodh Boddupalli, Jeff Moehlis
similarity 0.60
Abstract · Submitted to the SIAM J. Dynamical Systems

We present a machine learning framework for identifying sparse, interpretable models of dynamical systems directly from time-series data. Our approach parameterizes the underlying vector field using a neural architecture and trains it by minimizing a multi-step prediction loss over a finite horizon. To ensure numerical tractability, we optimize a mean absolute error objective averaged across prediction steps, and progressively increase the horizon during training. A key feature of this formulation is that it enforces consistency under repeated composition of the learned dynamics. As a result, the identified models exhibit significantly improved stability compared with approaches based on one-step regression of the vector field. When combined with sparsity-promoting regularization, this leads to parsimonious models that generalize beyond the training data. We demonstrate accurate recovery of systems exhibiting a wide range of behaviors, including stable and unstable fixed points, periodic orbits, and chaotic attractors. For chaotic systems, while long-term trajectory prediction is inherently limited by sensitivity to initial conditions, we show that multi-step training yields models with accurate short-term dynamics and strong agreement in long-time statistical properties, including mean, variance, and Lyapunov exponents. Moreover, we establish theoretical bounds linking trajectory error to statistical accuracy, providing a step toward a principled explanation for this behavior.

0.15

Conservation Buys Stability and Factoring Buys Counterfactuals in Physical World Models

cs.LG ·2026-09-17 ·Yufeng Wang, Parivesh Priye, Lu Wei, Haibin Ling
similarity 0.60
Abstract

A learned simulator can reproduce its training conditions accurately yet fail in two distinct ways once those conditions change. Over long rollouts, small errors accumulate until the trajectory drifts away from physically plausible behavior; under an intervention on a physical parameter, the model may continue to follow the law seen during training rather than the intervened one. We show that these two failures require different structural remedies. Evolving a learned energy with a symplectic integrator preserves the geometry of the conservative dynamics and keeps rollouts bounded and physically meaningful for up to $100\times$ the training horizon, while equal-capacity predictors, an energy-regularized predictor, and a tuned neural ODE diverge. By contrast, encoding the physical coupling through an explicit linear factorization enables the model to follow a never-seen sign of that coupling, whereas an unrestricted parameterization remains locked to the training law. Crucially, the two mechanisms are separable: removing the structure responsible for long-horizon stability leaves counterfactual transfer intact, while removing the factorized coupling destroys counterfactual transfer without eliminating stability. This double dissociation, established with matched controls that remove or replace one structural component at a time, persists beyond the headline three-body system and remains visible when the physical state must be inferred from pixels rather than provided directly. The result is a concrete design principle for physical world models: long-horizon stability and changed-law generalization arise from distinct structural commitments, and each can be imposed deliberately without requiring the other.

0.20

TEMPO: Regime-Aware Operator Learning with Locally Adapted POD Bases

cs.LG ·2026-11-16 ·Rodion Akinzhala, Alexander Terentyev, Alexey Golubev, Yury Maximov
similarity 0.60
Abstract

Operator learning approximates the solution map of a parametric partial differential equation from data, enabling fast prediction at new parameter values without repeated simulation. The dominant approach fits a single global low-rank basis, typically a proper orthogonal decomposition (POD), to all solution snapshots. This fails when the snapshots span qualitatively different solution shapes, from smooth diffusive profiles to near-discontinuous shocks. We propose TEMPO (Trajectory Expectation-Maximization Mixture of POD Operators), a two-phase framework for regime-aware operator learning. In the offline phase, TEMPO recovers latent regimes via expectation-maximization, fitting a dedicated linear or nonlinear basis to each discovered regime. In the online phase, a gating network and per-regime branch networks are trained jointly, routing new inputs to the regimes at inference. We test TEMPO on Burgers' flow, Sod shock tubes, Coanda flow, Darcy flow, Navier-Stokes flow and the heterogeneous Helmholtz equation. With 64 total POD modes, TEMPO gives 18.61% test error on Burgers', compared with 19.61% for global POD. It also gives lower errors on Coanda and Helmholtz.

0.24

Distillation of Foundation Models for Time-dependent PDEs

cs.LG ·2026-08-12 ·Daniel Musekamp, Boshra Ariguib, Andrei Manolache, Mathias Niepert
similarity 0.60
Abstract

Foundation models for time-dependent partial differential equations (PDEs) are trained on large and diverse collections of physical systems and can generalize effectively to new downstream tasks. After fine-tuning on only a few trajectories from a target domain, they can achieve strong accuracy in low-data regimes. However, these models are typically large and computationally intensive, limiting their usefulness as fast surrogates for numerical solvers. We propose Teacher Rollout Extension (TREX), a knowledge distillation framework that transfers the predictive capability of a pretrained foundation model into a compact and efficient student. Starting from a fine-tuned teacher, TREX augments limited downstream data by generating long synthetic trajectories through teacher rollouts, optionally with periodic noise injection. This procedure samples from the teacher-induced rollout distribution without requiring explicit knowledge of the initial-condition distribution, while exposing the student to long-horizon states and local recovery behavior around states encountered during autoregressive prediction. The student can further incorporate task-specific inductive biases, such as equivariance, that the teacher does not necessarily enforce. We evaluate TREX on multiple PDE benchmarks. The resulting students can match or surpass the teacher's accuracy while reducing the number of parameters by several orders of magnitude and achieving more than an order-of-magnitude speedup in inference.

0.34

On the Theoretical Limits of Learning Unconstrained Interatomic Potentials

cs.LG physics.chem-ph ·2026-10-10 ·Prajwal Ananth, Shuwen Yue
hot direction similarity 0.59
Abstract

Machine-learned interatomic potentials (MLIPs) accelerate molecular dynamics (MD) simulations by learning the energies and forces that govern atomic motion. Recent work has shown that relaxing physical inductive biases can improve computational efficiency and predictive accuracy, raising the question of whether energy-and-force matching alone is sufficient to ensure accurate sampling. We show that, for unconstrained hypothesis classes, the relationship between prediction error and sampling accuracy is exponentially ill-conditioned in dimension. In particular, prediction errors that decay exponentially with dimension can coexist with nearly maximal total variation distance between the learned and target distributions. We identify two failure mechanisms. The first is kinetic trapping, where relaxation to an accurate learned equilibrium requires exponentially long times, even when simulations are initialized from training configurations at which energies and forces are matched exactly. The second is equilibrium mismatch, where the learned stationary distribution itself remains far from the target. Under explicit assumptions, we show that equilibrium mismatch can persist even in local, symmetry-preserving models, where small per-atom errors coexist with a distributional discrepancy that approaches its maximum as system size increases. Finally, we establish global energy-control conditions that guarantee equilibrium accuracy and identify additional conditions required for rapid relaxation.

0.27

Long-Time Trajectory Approximation via SA-NODEs: Model Predictive and Floquet Strategies

cs.LG math.NA ·2026-08-11 ·Ziqian Li, Nikolaos M. Matzakos
theory similarity 0.59
Abstract

We study the approximation of dynamical systems by semi-autonomous neural ordinary differential equations (SA-NODEs) over long time horizons. For a single network trained on the whole horizon, the available error bound deteriorates double exponentially in the horizon length. We develop two training strategies that avoid this barrier, each built on a reset of the state. The model predictive strategy partitions the horizon adaptively and restarts every window from observed data: when training meets a prescribed tolerance on every window, the composite model meets it uniformly in time, with a parameter budget linear in the horizon for targets with a bounded, uniformly regular reachable tube. The Floquet strategy addresses autonomous targets with a stable limit cycle and uses no data at deployment: a certified contraction of the learned return map confines the error to linear growth in the number of elapsed periods. For the time-periodic architecture we deploy, the scalar certificate degenerates; we prove instead a uniform-in-time orbital guarantee whose hypotheses are measured on the trained model, and an obstruction showing that, for an exactly periodic learned field, small one-period error and a contracting stroboscopic map cannot hold at once. Numerical experiments on four benchmarks confirm the predicted error laws and measure the hypotheses of every guarantee.

0.04

Beyond Compression: Training Latent Representations for Stable Long-Horizon Rollout in Neural Surrogate Solvers

cs.LG cond-mat.mtrl-scics.CE ·2026-09-24 ·Andreas E. Robertson, Ashley T. Lenau, John D. Shimanek, Benjamin A. Jasperson, Vivek Oommen +3 more
similarity 0.59
Abstract

Latent neural surrogate solvers, or latent dynamics models, accelerate simulations of time-dependent physical systems by evolving a compressed latent space rather than resolving full-resolution fields directly. In principle this reduces computational cost and simplifies learning, but in practice errors often accumulate rapidly during long autoregressive rollouts, limiting predictive utility. We show that this instability does not stem from the latent representation itself, but arises when it is trained solely for reconstruction, producing representations poorly suited to long-horizon forecasting. We systematically evaluate training-level interventions that align latent representations with long-horizon rollout: Koopman operator learning and Hamming noise injection during autoencoder training to improve compression, together with noise injection and multi-step rollout fine-tuning to improve dynamics. Interventions that improve long-horizon rollout stability often degrade conventional training metrics, including reconstruction and one-step prediction accuracy. Collectively, these interventions reduce long-rollout error by approximately 40\% and match or exceed the accuracy of full-resolution models on two physics benchmarks, while requiring 2 orders of magnitude fewer floating point operations and half the GPU memory. Applied to mesoscale crystal-plasticity simulations of high-cycle fatigue, the resulting surrogate achieves stable extrapolation over horizons orders of magnitude beyond those observed during training. More broadly, these results show that neural compression should be designed not merely to reduce dimensionality, but to restructure the solution space for stable dynamical evolution, a key requirement for reliable, efficient neural surrogates in scientific applications.

0.24