CS Paper Radar

Warm-starting PDE solvers with any-dimensional machine learning

Wilson G. Gregory, George A. Kevrekidis, Ben Blum-Smith, Soledad Villar

cs.LG stat.ML ·submitted 2026-09-30 04:01 UTC · v1

Abstract

Any-dimensional machine learning models, such as graph neural networks (GNNs), can be naturally trained and evaluated on inputs of different sizes and dimensions. Inspired by the GNN transferability literature, we show mathematical conditions under which a partial differential equation (PDE) learning-based solver can be trained in small dimensions and directly applied to solve a higher dimensional PDE in a zero-shot fashion. These conditions are based on symmetries in both the partial differential equation and the initial data. When the equations satisfy the symmetries but the data does not, which is the case for many PDEs arising from physics, we show that our theory gives a principled way of warm-starting low-dimensional PDE solvers for higher dimensional PDEs. We apply this method on the heat equation, Burgers' equation, and the compressible Navier--Stokes equations, improving the performance in both zero-shot and typical training regimes on high dimensional data. For example, we train a surrogate model on 2D Navier--Stokes data and achieve better results on 3D test data than a baseline surrogate model trained on 3D data, while only using 12$\%$ of the flops and 20$\%$ of the total data size.

Comment
29 pages, 4 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers

cs.AI ·2026-10-02 ·Luis Medrano-Navarro, Giacomo Baldan, Qiang Liu, Benjamin Holzschuh, Jan Hagnberger +2 more
top venuecode/data similarity 0.76
Abstract · Accepted to NeurIPS 2026. Code and models are available at https://github.com/tum-pbs/GMP

Neural surrogate models for Partial Differential Equations (PDEs) on unstructured 3D geometries are often limited by poor generalization and the high cost of generating large-scale training datasets. Consequently, pre-training on massive datasets of related PDE dynamics has emerged as a critical alternative to enhance the robustness and scalability of these models. However, this strategy is neither compute- nor data-efficient, as it relies on massive pre-computed data that is very costly to generate. In this work, we introduce a disk-data-free pre-training framework tailored to both steady-state and transient regimes. For steady-state problems, we propose a geometry-driven strategy that leverages intrinsic shape descriptors to learn representations of complex 3D domains. For transient problems, we introduce a physics-driven approach based on online generation of synthetic PDE data, enabling scalable pre-training without reliance on expensive datasets. Across multiple experiments, our approach achieves faster convergence, greater data efficiency, and higher accuracy during fine-tuning, particularly under realistic low-data regimes. This methodology provides a practical pathway toward data-efficient neural emulators for large-scale simulations. Code and models are available at https://github.com/tum-pbs/GMP.

0.64

The limits of exactness: On the failure of automatic differentiation in physics-informed machine learning

cs.LG ·2026-09-27 ·Ameya D. Jagtap
similarity 0.70
Abstract · 18 pages, 2 figures

Automatic differentiation (AD) lets neural networks compute derivatives of governing equations to machine precision, and this precision has made it the computational backbone of physics-informed machine learning. Yet exactness in the mathematical sense is not the same as fidelity to the physics. Here I argue that a derivative can be numerically perfect and still be the wrong derivative for the problem at hand, because AD, by construction, has no notion of the physical structure a solution must obey. Convection and its associated directionality, diffusion, and dispersion are only the most visible instances of a much longer list that spans all branches of computational science and engineering, including conservation, thermodynamic consistency, symmetry, symplectic structure, positivity, monotonicity, and boundedness. Recognizing this broader gap reframes how the field should build the next generation of PDE-driven neural surrogates.

0.15

Learning PDE Time-Stepping with Neural Cellular Automata

cs.LG stat.ML ·2026-08-31 ·Esha Saha, Hao Wang
benchmark similarity 0.69
Abstract · 4 figures, 7 tables

Classical numerical solvers for partial differential equations (PDEs) are computationally expensive to solve repeatedly across varying initial conditions, motivating the need for learned surrogates. In this paper, we propose a trainable Neural Cellular Automata (NCA) based surrogate model for learning long time PDE dynamics. Rather than mapping an entire initial field to a full trajectory in one shot, our proposed model learns a small, local, homogeneous update rule that is applied identically and repeatedly at every grid cell, mirroring the locality of differential operators. We benchmark this framework against three baselines: PDE - Net, a modified physics-informed neural network (PINN), and a Fourier Neural Operator (FNO), on five canonical PDEs (heat, advection, Burgers, Allen - Cahn, and Fisher - KPP), evaluated at temporal domain two times beyond the training temporal domain. The proposed model achieves the lowest long-horizon relative errors on the majority of the experiments.

0.06

Solving and Learning Navier-Stokes Dynamics with the Neural Basis Method

cs.LG physics.flu-dynmath.NA ·2026-10-23 ·Yuhe Wang, Zhu Wang, Min Wang
similarity 0.67
Abstract · 23 pages, 6 figures

Physics-informed machine learning has become a major direction in predictive modeling of fluid dynamics, combining governing physics with learned representations. Yet prevailing formulations neither guarantee that small PDE-residual losses imply small solution errors nor scale effectively with increasing problem size. Here we present the neural basis method (NBM) as a new approach for Navier--Stokes dynamics. NBM is built on a ``representation-then-projection'' principle: fixed neural bases define the representation space, while projection of the governing equations determines the expansion coefficients. The same bases define common coordinates for the parametric solution manifold, recasting operator learning as a parameter-to-coefficient mapping and enabling scalability by removing explicit spatial dependence. For incompressible flow, NBM yields a velocity-only formulation whose residual provides an a posteriori velocity-error estimate, establishing a certified, data-free training objective for coefficient learning. Theoretical analysis and numerical experiments demonstrate the accuracy, efficiency, generalization, and scalability of NBM, with substantial improvements over baselines.

0.19

ALETHEIA: A Multi-Frequency Eddy Current Pulsed Thermography Dataset for Neural Operator Learning in Nondestructive Testing

cs.LG ·2026-11-08 ·Changbin Sun, Jie Xiao, Xiaotian Chen, Yuankai Wu
benchmarkdatasetcode/data similarity 0.67
Abstract · 62 pages, 19 figures

Learning neural solvers for spatiotemporal partial differential equations (PDEs) under real-world constraints remains a key challenge in scientific machine learning, especially for inverse tasks with sparse and noisy boundary observations. We present the Aletheia dataset, the first 3D benchmark for learning data-driven solvers in the context of nondestructive testing (NDT). The dataset simulates eddy-current-induced heating in conductive solids and models the resulting transient heat propagation governed by the heat equation. Aletheia contains over 4,700 high-resolution samples across 10 excitation frequencies (1-100,kHz), each providing volumetric heat source and temperature fields over time. It supports both forward prediction of temperature evolution and inverse reconstruction of internal heat sources or defects from surface infrared measurements. Real infrared thermography data from cracked rail specimens are included for calibration and comparison. We define four canonical tasks on both regular and irregular grids and benchmark them using various neural operators. Aletheia establishes a unified platform for evaluating neural PDE solvers under realistic NDT conditions, enabling progress in reliable, data-driven inverse modeling. The dataset and code are publicly available at https://github.com/sherlock-0327/ALETHEIA.

0.42

PDE-State Conditioning for Learned Optimization Using Physics-Informed Neural Networks

cs.LG ·2026-11-18 ·Farzana Yasmin, Ricardo Vilalta
similarity 0.66
Abstract

Physics-informed neural network (PINN) optimization depends on global properties of the underlying PDE and how the physics residual evolves across the domain. Standard optimizers such as Adam use fixed gradient-based statistics, while learned PINN optimizers replace the fixed rule with a learned update built from local parameters and optimization-history features. However, none explicitly use PDE-level information. We study two coupled design questions: what PDE-level information should a learned optimizer observe, and how should that information influence its update? We introduce an 11-dimensional PDE state-capturing operator structure, residual geometry, and short-horizon update response, and use it to condition a structured meta-learned optimizer through bounded feature-wise modulation. Repeated design studies identify the compact residual-geometry state G2 and bounded FiLM as the strongest overall choices among those tested. The resulting PDE-State-conditioned optimizer improves all evaluation metrics (relative-L2, L1, solution MSE, and physics MSE) over Adam and the unconditioned learned optimizer on Linear Advection, KdV, and Burgers.

0.18

Structure-preserving fidelity diagnostics for partial differential equations

cs.LG physics.comp-ph ·2026-10-12 ·Kyriakos Flouris
similarity 0.66
Abstract · Preprint

Machine learning surrogates of partial differential equations (PDEs) are commonly trained on data generated by numerical solvers. Therefore, the training data obey not the PDE but the modified equation of the numerical scheme, and the test loss, computed on data from the same solver, cannot distinguish a model that learned the physics from one that learned the discretization error. Here, we show that the difference is nevertheless visible in the trained model. We propose a suite of structure-preserving fidelity diagnostics, simple scores that measure whether a learned model respects one known property of the physics, such as a conserved quantity, a symmetry or a Hamiltonian structure, without access to the training data or the governing equation. Over five systems, the diagnostics track the error with respect to the true dynamics with a rank correlation of up to $0.94$, while the test loss is uncorrelated or anti-correlated with it. For the Burgers, Korteweg-de Vries and Navier-Stokes equations, the diagnostics separate numerical schemes that the test loss ranks as equivalent, including schemes of the same order. order.

0.16

Interpolating Neural Operator (INO): A Data-Free and Efficient Approach for Learning PDE Solution Operators

cs.CE ·2026-09-29 ·Jiachen Guo, Ye Lu, Naichen Shi, Thomas J. R. Hughes, Wing Kam Liu
benchmark similarity 0.66
Abstract

Neural operators have become a popular approach to approximate the solution operators of parametric partial differential equations (PDEs). However, existing neural operators either require a large amount of simulation data or a long physics-informed training on GPUs, and they cannot tell how accurate an individual prediction is. In this paper, we propose the Interpolating Neural Operator (INO), a data-free interpolating neural network that is trained directly on the weak form of the PDE. In INO, the Karhunen-Loève coordinates of the input field are treated as additional inputs together with the spatial coordinates, and each input is approximated by a C-HiDeNN sub-network whose trainable parameters are nodal values. Since the network is multilinear in its parameters, training reduces to a sequence of one-dimensional linear solves by greedy alternating least squares. As a result, INO trains on one CPU core and predicts a new solution in microseconds. For coercive problems, the total error of every prediction is bounded by a computable residual bound that requires no reference solution, and the same bound applies to the predictions of other methods that satisfy the boundary conditions exactly. Before each prediction, INO checks whether the leading coordinates of the input lie within the range on which it is trained, and inputs outside this range can be passed to a conventional solver or to an INO trained on a wider range. INO is compared with physics-informed FNO and DeepONet on different benchmarks. INO is the most accurate model on most of these problems, by 15$\times$ on two-dimensional Helmholtz at $65^2$ and 53$\times$ on the diffusion-reaction benchmark, and on the one- and two-dimensional problems its training on one CPU core takes 3-80$\times$ less time than the physics-informed baselines on one GPU.

0.20