CS Paper Radar

LPINNs: First-Layer Gated Localization for Physics-Informed Neural Networks

Lakshay Chawla, Hardik Jain

cs.LG cs.CEcs.NE ·submitted 2026-09-19 12:36 UTC · v1

Abstract

Physics-informed neural networks (PINNs) use one shared representation over the computational domain, which can become difficult to optimize on long domains and for high-order operators. We study a minimal alternative: multiply the first hidden activation of an otherwise unchanged dense PINN by input-dependent localization functions, giving first-layer units receptive fields without partitioning the domain or adding interface losses. We screen 13 families of localization functions, in up to three parameterizations each, on a nonlinear harmonic oscillator (HO), a heat equation on a long spatial interval, and a manufactured four-dimensional (4D) fourth-order problem, with ten paired seeds throughout. Three configurations give large reductions in solution error at matched budgets: (i) Fixed Gaussian localization functions on the $2π$ HO domain cut mean solution RMSE from $4.8369\times10^{-1}$ to $8.83\times10^{-3}$ at 3k epochs. (ii) The inverse-quadratic family with learnable centers and widths cuts it from $2.896\times10^{-1}$ to $3.06\times10^{-2}$ on the $8π$ heat domain at 10k epochs. (iii) Fixed bump localization functions cut it from $1.75947\times10^{1}$ to $2.260\times10^{-1}$ on the $4π$ 4D domain at 10k epochs. Every paired seed improves in these three comparisons. The screen also shows that the mechanism is not a free win: on HO only 2 of 13 families beat the baseline, and 10 of the remaining 11 are 9 to 23 times worse; on 4D four families are non-finite and five are more than three orders of magnitude worse than the baseline. The inverse-quadratic family is the only one that beats the baseline on all three equations. Overall, these results show that first-layer localization can provide measurable improvements to baseline PINNs on long-domain and high-order problems.

Closest papers

Nearest neighbours by meaning across the whole index.

Equidistribution of Solution Gradients in Physics-Informed Neural Networks via Coordinate Transformations

cs.AI math.NAcs.LG ·2026-11-06 ·Tianchi Yu, Ivan Oseledets
similarity 0.68
Abstract

Physics-Informed Neural Networks (PINNs) often struggle to approximate solutions with sharp gradients due to optimization difficulties and spectral bias. Motivated by the principle of gradient equidistribution from the stretching approach in grid generation, which redistributes solution gradients to reduce functional complexity, we propose Coordinate Transformation PINNs (CTPINNs), which incorporate learnable coordinate transformations to map the physical domain into a computational domain where the physical solutions become approximately affine, making them easier for neural networks to learn. We propose two constructions of the transformation: an explicit construction based on Majorants and an implicit construction based on invertable neural networks. Experimentally, the explicit construction can improve accuracy for PDEs with available explicit coordinate transformations, and the implicit construction can be combined with other advanced PINN models as an efficient plug-in module to improve performance across a range of challenging PDE benchmarks.

0.21

Linearized PINN with pretrained nonlinear layers

cs.LG math.AP ·2026-09-14 ·Wenhao Chen, Alexandre M. Tartakovsky
similarity 0.65
Abstract

We propose a linearized Physics-Informed Neural Network (lPINN), a reduced-order neural basis method for forward and inverse differential equations. In an offline stage, lPINN learns operator-compatible continuous neural basis functions from an ensemble of numerical solutions. The basis functions are differentiable through automatic differentiation and are pretrained using solution data together with either derivative information or physics residuals. For each new problem instance, the basis functions are frozen and the solution is obtained by minimizing the governing-equation residual together with applicable initial, boundary, regularization, and observational terms. Unlike surrogate and operator-learning methods, the training data define the trial space offline, while the instance-specific solution is computed online by enforcing the governing physics. Relative to vanilla PINNs, lPINN pretrains the nonlinear hidden-layer representation offline and performs online inference only in the final linear layer. We evaluate lPINN on forward and inverse problems for the advection-diffusion equation, Burgers' equation, and the nonlinear pendulum equation. Compared with vanilla PINNs, lPINN achieves lower solution and parameter errors while reducing online inference times by approximately one to more than three orders of magnitude, with the largest gains generally observed for limited residual or measurement data. Cross-resolution experiments show that the learned continuous representation can be evaluated on finer meshes without retraining and with nearly unchanged accuracy.

0.22

The importance of tensor handling and implementation of physics-informed DeepONets

cs.LG physics.comp-ph ·2026-10-15 ·Nahum Banks, Gustavo A. M. de Almeida, Sergio Maldonado
similarity 0.65
Abstract

Physics-informed neural networks (PINNs) reduce or eliminate the need for large labelled datasets by enforcing governing equations during training. Physics-informed Deep Operator Networks (PI-DeepONets) extend this approach to learning solution operators, enabling rapid prediction across families of problems. Despite these advantages, we show that their application to higher-dimensional problems remains limited by the computational cost of training. To address this issue, we propose a modified PI-DeepONet architecture that reduces this cost by removing redundancy in how the Branch network processes input functions across collocation points. The proposed formulation is mathematically equivalent to the original architecture but reduces input tensor sizes and the number of Branch network evaluations. We analyse the computational complexity of both architectures and derive theoretical scaling laws for training time with respect to the number of collocation points and input functions. The proposed architecture is evaluated on 1D, 2D, and 3D linear advection problems. In all cases, it achieves comparable predictive accuracy to the original PI-DeepONet while consistently reducing training time. In 2D experiments, a maximum speed-up of $3.75\times$ is observed, with empirical training-time scaling agreeing with the predicted linear dependence on collocation points and input functions. Furthermore, training a 3D model on an NVIDIA A100 GPU is only feasible using the proposed architecture, as the original formulation exceeds memory limits. These results demonstrate that careful consideration of how PI-DeepONets are implemented can yield substantial gains, enhancing the practical performance of PI-DeepONets and enabling their application to otherwise intractable physics-informed learning problems.

0.12

Newton-PINet: A fast physics-informed neural network with Newton linearization for meta-learning nonlinear PDEs

cs.LG cs.AIcs.CEphysics.comp-ph ·2026-11-16 ·Yuchen Fan, Chang Wei, Pao-Hsiung Chiu, Chin Chun Ooi, Heyang Wang +1 more
top venuebenchmarkcode/data similarity 0.64
Abstract · Accepted at NeurIPS 2026

Scientific machine learning has opened new avenues for solving parameterized partial differential equations (PDEs), enabling models to learn a family of PDEs and generalize to unseen instances. In this context, data-driven operator learning methods typically require large training datasets, while physics-informed neural networks (PINNs) suffer from difficult optimization and limited generalization, especially for nonlinear PDEs. We propose Newton-PINet, a physics-informed network enhanced by Newton linearization, offering an effective meta-learning framework for nonlinear PDEs. Newton-PINet (i) employs a physics-informed multilayer network with skip connections, where the output-layer weights are solved by least squares; (ii) adopts a two-stage learning strategy that first leverages gradient-based training to learn robust representations from the available training tasks, and then performs gradient-free fine-tuning on the output layer for fast task-specific generalization; and (iii) incorporates a Newton linearization method to speed up the least-squares iteration for nonlinear PDE problems. On a challenging nonlinear reaction-diffusion benchmark, Newton-PINet achieves up to three orders of magnitude lower relative error than recent neural solvers, while using 16x fewer training tasks and over an order of magnitude less training time (about 5 minutes versus several hours). This work advances the meta-learning of PINNs toward data-efficient, fast, and generalizable physics solvers. The code and datasets are publicly available at: https://github.com/fycAIwj/Newton-PINet.

0.50

Learning Physics from an Imperfect Ancestor

cs.LG ·2026-09-21 ·S. Mohammad Mousavi, Teeratorn Kadeethum, Nikolaos Bouklas, Somdatta Goswami
similarity 0.63
Abstract

Neural operators evaluate parametric partial differential equations cheaply but degrade sharply outside their training distribution. Physics-informed neural networks avoid dependence on labeled data, yet their optimization can be basin-fragile: when the governing residual admits multiple solutions, a PINN trained from scratch may converge to a physically incorrect state despite achieving a small residual. We show that these failure modes can be addressed jointly: an imperfect NO provides the structural prior needed to place a PINN in the correct solution basin, while the PDE residual refines the solution beyond the operator's accuracy. We introduce a three-stage framework that freezes the spatial basis of a physics-informed NO, extrapolates its solution branch to an out-of-distribution parameter using a polynomial continuation prior, and distills the resulting field into a fresh PINN. The NO need not be accurate at the target; it transfers solution-branch information, while PDE residual minimization in the PINN governs convergence. We evaluate the framework on three nonlinear PDEs: 1D viscous Burgers, 2D steady Allen-Cahn near a pitchfork bifurcation, and 2D steady lid-driven cavity flow. For Allen-Cahn, where the trivial solution satisfies the PDE residual exactly, a standard PINN collapses to the trivial zero branch, whereas distillation from the crude extrapolated operator recovers the non-trivial branch that matches the finite-difference reference. For the lid-driven cavity, extrapolating to a Reynolds number of Re = 3200 accelerates convergence to the correct physical state, achieving competitive accuracy using fewer parameters and optimization steps than recent literature baselines. These results establish a simple principle: an NO need not accurately predict the solution to be useful; it only needs to identify the correct basin from which PINN optimization can recover it.

0.12

Equation informed time reparametrization for physics informed neural networks

cs.CE ·2026-10-30 ·Orestis Belokas, Christoforos Rekatsinas
similarity 0.63
Abstract · 25 figures, 4 tables

Physics Informed Neural Networks offer a mesh-free approach to solving differential equations, but their application to time-dependent problems remains challenging, particularly when the governing dynamics involve widely different time scales. With collocation points spread uniformly in time, short intervals of rapid change are under resolved. This work introduces an equation informed time reparameterization framework that modifies the temporal coordinate used during PINN training based on the local dynamics of the governing equation. The proposed method follows a two stage procedure. First, a pilot PINN training provides an approximate solution, which is used to linearize the governing differential operator and estimate its local response to small perturbations. These responses are combined into a time-dependent rate function, large where the dynamics are fast, that defines a strictly increasing transformation from physical time to a new computational time coordinate. A second PINN is then trained on the transformed governing equation, with uniformly sampled collocation points in the new coordinate producing a nonuniform distribution in physical time. Different combination rules and spatial reduction strategies are considered. With the cost of the pilot training included, the method reduces the error by a factor of 4 to 29 relative to the best alternative method on the Kachanov Rabotnov creep-damage model near rupture, and by a factor of 5 to 16 relative to every alternative method on the Frank-Kamenetskii equation near blow up.

0.08

A Systematic Analysis of a Small Optimal Subspace for Physics-Informed Neural Networks: $n$-DAS $\subset \mathbb{R}^p$

cs.LG math.APmath.OC ·2026-11-18 ·Ahmad AliAhmad, Khemraj Shukla
position similarity 0.62
Abstract · 57 pages, 17 figures, 7 tables. In the title, n-DAS denotes an optimizer of order n restricted to the dominant aligned subspace (DAS): 1-DAS…

The search for a low-dimensional subspace that governs neural network training has been a persistent goal in optimization. Yet subspaces defined by Hessian eigenvalue magnitude, such as the dominant/bulk decomposition underlying Dom-SGD and Bulk-SGD, have repeatedly failed to capture where optimization takes place, and adaptive Krylov-based alternatives built on energy-fraction thresholds show promise but still struggle at lower eigenmodes. We argue this failure is structural: the directions of highest curvature are not, in general, the directions along which training moves. We introduce the dominant aligned subspace (DAS), constructed from a full eigendecomposition of the Hessian but selected by gradient alignment rather than eigenvalue magnitude, as a tool for understanding optimization dynamics and a foundation for designing methods that exploit them. To expose this structure under the hardest conditions, we use physics-informed neural networks (PINNs) as a testbed: differential operators from the parabolic, elliptic, and hyperbolic classes induce severely ill-conditioned, anisotropic loss landscapes. Across Poisson, heat, wave, and viscous Burgers benchmarks DAS-restricted training with a line search matches full-space training even where the projected Hessian remains ill-conditioned, showing empirically that convergence does not require all Hessian eigendirections but only a low-dimensional, gradient-aligned subset. We extend the analysis to Gauss-Newton within the DAS and discuss what this structure implies for future subspace-aware optimizers.

0.32

When Does Frequency Decomposition Benefit Physics-Informed Neural Networks? A Preliminary Ablation Study

cs.LG cs.AI ·2026-08-24 ·Shubham Rai
benchmarkhot direction similarity 0.62
Abstract

Partial differential equations (PDEs) often have high-frequency and multi-scale features that neural networks struggle to approximate. Physics-Informed Neural Networks (PINNs) build the governing equations directly into training, but suffer from spectral bias: they learn low-frequency components faster than high-frequency ones. Techniques such as Fourier feature embeddings and sinusoidal activations address this, but most studies assume they help across the board without checking which spectral regimes actually benefit. We introduce a dual-branch, spectrally-gated architecture (DBSG-PINN) that splits low- and high-frequency components into separate subnetworks joined by an adaptive gate, and use it to run a partially controlled ablation of frequency decomposition and spectral routing. We test this on five one-dimensional benchmark PDEs, ranging from smooth, single-scale problems to oscillatory, multi-scale ones. Frequency decomposition helps most on the spectrally complex benchmarks, cutting relative $L_2$ error by up to $59.2\%$ on a multimodal wave problem, but gives little benefit on smoother PDEs. On one benchmark (1D Wave), it performs substantially worse than a simpler fixed-combination variant. The gate's benefit scales with how spectrally rich the target solution is: the full model's advantage over the ablations is largest on multi-scale benchmarks and smallest (or negative) on single-scale ones, consistent with the gate exploiting frequency structure rather than acting as noise,though we do not directly visualize or quantify its spatial activations in this study. All results come from a single training seed across five 1D benchmarks, so we present this as an exploratory study meant to raise questions rather than answer them, and outline the additional seeds and benchmarks needed to test whether the pattern holds.

0.32