CS Paper Radar

Parallel Sparse Identification of Nonlinear Dynamics in Python: Task- and Data-Parallel SINDy with Numba, OpenMP, Threads and Multiprocessing for the Lorenz and Rössler Systems

Pablo Guillen-Rondon

cs.DC cs.MS ·submitted 2026-11-03 17:01 UTC · v1
code/data

Abstract

The sparse identification of nonlinear dynamics (SINDy) recovers interpretable governing equations from time-series data by sparse regression over a library of candidate functions. As data sets, candidate libraries and the number of repeated fits grow, the cost of SINDy becomes dominated by the evaluation of the library matrix and its least-squares problems. We present an open-source Python implementation of SINDy in which a single set of loop-based kernels is just-in-time compiled with Numba and executed on a shared-memory multicore node by four interchangeable back ends: Numba prange, OpenMP directives through PyOMP, Python threads that release the global interpreter lock, and multiprocessing with shared memory. The normal equations are accumulated in one cache-blocked pass over the data, with an automatic fallback to a tall-skinny QR reduction for ill-conditioned problems. Two decompositions are studied on the Lorenz and Rössler systems: an ensemble (task-parallel) workload of 960 independent identifications and a library (data-parallel) workload of one trajectory with $2\times10^7$ samples and a fifth-order library of 56 candidate terms. All back ends recover the exact sparsity pattern in every identification, with coefficient errors identical to PySINDy. On a dual-socket Intel Xeon E5-2650 v4 node of the University of Houston-Downtown HPC cluster (24 cores, 48 hardware threads), the compiled serial code is 4.3-5.6x faster than PySINDy, the parallel back ends reach speed-ups of 14.8-20.3 on 24 cores (parallel efficiency 0.61-0.84), and the overall time to solution is reduced by up to 117x with respect to PySINDy. We analyse the differences between the back ends, the effect of the two sockets and of hyper-threading, and give practical recommendations.

Comment
10 pages, 3 figures, 2 tables. Code and data: https://github.com/guillenrondonp/parallel-sindy

Closest papers

Nearest neighbours by meaning across the whole index.

Tensor-Train Weak SINDy: Identifying High-Dimensional Nonlinear Dynamics

cs.LG cs.CEstat.ML ·2026-09-08 ·Will Houser, Vanja Dukic, David M. Bortz
similarity 0.65
Abstract · 34 pages, 8 figures

Weak Sparse Identification of Nonlinear Dynamics (WSINDy) provides a noise-robust approach for learning dynamical systems from data without requiring numerical differentiation. However, for high-dimensional systems, tensor-product libraries of candidate functions grow exponentially with the state dimension, making standard WSINDy expensive in both computation and memory. The Multidimensional Approximation of Nonlinear Dynamics (MANDy) addresses this scaling through a tensor-train (TT) representation of the candidate library, but does not provide a mechanism for sparse model selection. Here, we combine these approaches to develop TT-WSINDy, which performs the weak-form transformation, regression, and sparsification in TT format. We show that the TT formulation recovers the corresponding WSINDy regression problem and derive polynomial time and memory complexity bounds for the tensor-train sparsification procedure. Numerical experiments demonstrate robustness to measurement noise and computational savings for high-dimensional systems.

0.19

Sparse Orthogonal Regression Technique: A Spectral Framework for Equation Discovery, Approximation, and Integration

cs.LG ·2026-08-13 ·Sabin Roman, Ljupco Todorovski, Saso Dzeroski
published similarity 0.56
Abstract · 15 pages, 4 figures. Accepted for oral presentation at the 29th International Conference on Discovery Science (DS 2026), Mainz, Germany, Oct…

We develop the Sparse Orthogonal Regression Technique (SORT), a sparse spectral framework for learning orthonormal-basis expansions from noisy and irregularly sampled data. SORT estimates expansion coefficients directly from observations using L1-regularized regression, avoiding explicit quadrature or analytic inner-product evaluation. The central application is data-driven discovery of ordinary differential equations: vector fields are represented in chosen orthogonal bases and learned as sparse coefficient expansions. This provides a complementary route to symbolic regression, grammar-based discovery, and SINDy-style sparse identification by first recovering a compact spectral representation, which can later guide searches for simpler analytic forms. Across the dynamical-system experiments, SORT matches or improves upon library-based sparse-regression baselines when the basis is well adapted to the problem, and shows more stable degradation under sparse sampling, noisy derivative estimates, and representation mismatch. Specific examples illustrate why this representation is useful: if a finite library misses the problem-specific nonlinearity, the resulting model can fail. SORT is not immune to mismatch, but it shifts the problem away from brittle selection among generic terms to basis design adapted to the problem domain. The experiments also show that dominant low-order coefficients persist as model order increases, supporting order-consistent model growth. Beyond equation discovery, the same learned expansion supports nonlinear approximation and estimation of complex, high-dimensional integrals by coefficient readout. Overall, SORT provides a reusable intermediate representation for system identification, approximation, and integration, while making basis design an explicit part of the scientific modeling problem.

0.28

Scaly: an Optimizing Compiler for Optimal Control

eess.SY math.OC ·2026-10-30 ·Tudor A. Oancea, Colin N. Jones
benchmarkcode/data similarity 0.55
Abstract

We present Scaly, an open-source optimizing compiler for optimal control, written in Python, that generates fast C code. Scaly provides a modeling language embedded in Python for the parametric optimization problems that arise in model predictive control and moving horizon estimation, and computes the sparse derivatives that standard optimization solvers need. Scaly accepts general nonlinear programs but applies optimizations specific to optimal control, which preserve the repeated structure of a problem, such as the stages of a horizon, through differentiation down to the generated loops. The generated C depends only on the C math library and never allocates memory dynamically. On three benchmark problems, Scaly evaluates the Hessian of the Lagrangian up to 26 times faster than CasADi, with smaller binaries and build times that stay nearly constant with the horizon. With the same solver, Scaly shortens the solver time per control step on every benchmark, by a factor of up to 9.

0.43

Kernel-Guided Learning of Lifted Sparse Polynomial Models of Nonlinear Dynamical Systems

eess.SY cs.LG ·2026-11-15 ·Mohammad Itani, Timm Faulwasser
theory similarity 0.55
Abstract

Polynomial models of dynamical systems enable constructive analysis via sum-of-squares programming. Yet, many physical systems involve non-polynomial nonlinearities which for certain classes of systems may be inferred from the underlying physics even when the exact dynamics are unknown. We show that fitting lifted polynomial models directly to lifted data is problematic, since the exact lifting constraints render the lifted derivatives linearly dependent. Instead, we learn the dynamics in the extended state, i.e., the state together with the known nonlinearities, and obtain the dynamics of the nonlinearities via the exact lifting relations. To avoid the combinatorially large libraries that sparse regression requires when the polynomial degrees are unknown, we propose KINDy, which selects the degree of each component by kernel ridge regression before applying sparse regression to the correspondingly reduced library. We prove that, for sufficiently small noise and regularization, provided the data satisfy a rank condition, the selected degree never underestimates the true degree. On the cart-pole system, KINDy recovers the true degrees at low noise levels and achieves speedups of up to 8.5 over sparse regression on the full library, while in almost all cases yielding models that are at least as sparse and have lower or equal test errors.

0.17

py-ARGOS: A Python package for automatic, statistically principled discovery of ordinary differential equations from data

cs.MS ·2026-10-25 ·Yuzheng Zhang, Rui Carvalho
benchmark similarity 0.54
Abstract · 17 pages, 4 figures

Recovering the governing equations of a dynamical system from data is a central problem in data-driven science, yet the available Python tooling forces a compromise among automation, statistical rigor, and reliability. We present py-ARGOS, a Python package implementing two automatic algorithms, bootstrap ARGOS and Bayesian-ARGOS, behind a uniform interface. Both are statistically principled throughout: automated model selection reduces the candidate library, each coefficient is reported with a bootstrap confidence or Bayesian credible interval that quantifies its uncertainty, and a term is retained when this interval excludes zero. Built-in diagnostics additionally flag unreliable fits. On chaotic benchmark systems, the two algorithms improve on the SINDy family's identification performance.

0.28

Parallel Sparse Outputs for Tensor Compilers

cs.DC ·2026-10-13 ·Adrian Gushin, Pau Sum, Willow Ahrens
similarity 0.53
Abstract

Sparse tensor compilers are commonly used high-performance tools that optimize tensor algebra programs over notoriously complex sparse data structures. Despite the ever-increasing size and scale of sparse datasets, there exists astonishingly little research on developing a parallel sparse tensor compiler. Existing approaches fail to parallelize or make assumptions about access patterns or output formats that limit generality. This paper presents Birdseed, the first tensor compiler to support all loop orderings over any standard sparse format in parallel. More specifically, Birdseed supports parallel sparse scatters and directly writes into list, bytemap, hash, and dense levels of fibertree sparse formats. Our approach exhibits asymptotic memory and time improvements over all previous work. We additionally demonstrate empirical speedups on challenging kernels to parallelize, such as histogram construction and SpMSpV.

0.23

Data-Driven Sparse Identification and Robust Tube-Based Predictive Control for Multivariable Chaotic Systems

eess.SY ·2026-11-13 ·Shahin Razani
theory similarity 0.53
Abstract

This paper addresses the data-driven synchronization and robust control of multivariable chaotic systems subject to strict actuation constraints, modeling uncertainties and bounded input disturbances. Using empirical data collected from a four-dimensional chaotic Rossler response system, Sequentially Thresholded Least Squares (STLSQ) sparse regression is employed to recover a nominal control-affine model, achieving an identification accuracy of R^2 = 1.0000 across all channels with root-mean-square error within [4.15e-6, 8.33e-4]. To establish a mathematically rigorous bridge between data-driven identification and robust synthesis, all structural discrepancies in the drift vector field and input matrix are unified into a single lumped uncertainty field delta(x,u). A computable scalar bound delta_bar is established based on library truncation error, regression residuals, input bounds, and Lipschitz constants (Lemma 1). We subsequently prove that input-output feedback linearization maps delta(x,u) into a state- and control-dependent additive disturbance d(z,nu) in Brunovsky canonical coordinates, bounded by ||d(z,nu)||_2 <= alpha_1 delta_bar + alpha_2 delta_bar ||nu||_2 with closed-form expressions for alpha_1 and alpha_2 (Theorem 1). This certified bound, together with a bounded input disturbance and a zero-order-hold margin, yields a compact disturbance set that seeds a robust tube-based model predictive controller. We prove that the resulting tightened quadratic program is recursively feasible, that the physical actuator limit (|u_i| <= 1100) remains inactive under an explicit sufficient condition, and that the closed-loop tracking error is Input-to-State Stable (ISS) with an asymptotic gain that is linear in delta_bar plus a term due to the exogenous input disturbance (Theorems 2 and 3).

0.24

Symbolic-Permutation-Based Direct Sparse Jacobian Assembly for Accelerated Power-Flow Analysis

eess.SY cs.PFmath.NA ·2026-10-08 ·Tianshi Cheng, Venkata Dinavahi
code/data similarity 0.53
Abstract · 9 pages, 3 figures

The repeated assembly of the sparse Jacobian matrix constitutes a primary computational bottleneck in the Newton-Raphson power-flow analysis. Through an algebraic equivariance analysis of the Jacobian matrix, this paper proposes a direct sparse Jacobian assembly method based on symbolic permutation. By pre-applying the permutation to the admittance matrix during a one-time symbolic phase, the runtime bus-type projection overhead is entirely eliminated. Consequently, complex Jacobian differentiation and global matrix assembly fuse into a single operation, yielding a single-pass, branchless algorithm that streams values directly into contiguous memory. The framework is realized in the open-source RustPower solver. Benchmarks on IEEE test systems and PEGASE 9241 demonstrate that the proposed assembly is 24.4-30.1x faster than the MATPOWER- style baseline and 6.0-7.8x faster than the Schafer-Braun optimized assembly. End-to-end, the grid-level solve time is 11.7x-154.3x faster than pandapower 3.5.4 with Numba, while leaving numerical results and convergence behavior unchanged. Compared with the native C++ LightSim2Grid 1.0.0, grid-level performance is comparable on IEEE 39 and 1.3x-1.8x faster on IEEE 118 and the PEGASE systems. The method extends directly to general problems where a sparse Jacobian inherits its structure from a static network operator, including transient stability analysis, EMT simulation, and OPF Hessian assembly.

0.62