CS Paper Radar

Convex Equilibrium Potentials for Monge--Ampère: Curvature Bounds and A Posteriori Evaluation

Edison Vázquez, Yingdong Lu, Tomasz Nowicki

cs.LG math.NA ·submitted 2026-10-22 14:53 UTC · v2

Abstract

Equilibrium neural networks define their hidden state as the fixed point of a contraction, so their effective depth is set at evaluation time rather than by the architecture. Using them for fully nonlinear PDEs raises two issues that explicit networks avoid: second derivatives may grow with the effective depth, and the potential is only ever evaluated approximately. We study nonnegative softplus equilibrium networks with contraction factor $q<1$, a convex class suited to the Monge--Ampère equation. A positivity cancellation in the resolvent shows that the Hessian grows at most like $(1-q)^{-2}$, rather than the $(1-q)^{-3}$ of a direct estimate, and an explicit one-neuron family shows that this exponent is sharp; third derivatives satisfy a sharp $(1-q)^{-3}$ bound. Residual-based a posteriori estimates then control the state, its input derivatives and, in two dimensions, the Hessian determinant at an approximate equilibrium. Combined with interval arithmetic on complete covers of the domain, they yield rigorous uniform bounds on the Monge--Ampère residual and, through a comparison argument, on the solution error of a fixed trained network. On two manufactured Dirichlet problems on the unit disk, all six trained models are certified with residual below $0.25$ and boundary error below $0.01$. Adaptive stopping halves float64 evaluation time at best, whereas certified adaptive evaluation remains costlier than fixed depth.

Closest papers

Nearest neighbours by meaning across the whole index.

Residual neural networks overcome the curse of dimensionality for semilinear heat equations

cs.LG math.NAmath.APmath.PR ·2026-09-03 ·Ilkhom Mukhammadiev, Diyora Salimova
theory similarity 0.60
Abstract · 29 pages, 1 figure

Rigorous results show that feedforward neural networks can overcome the curse of dimensionality in the numerical approximation of high-dimensional partial differential equations (PDEs), but comparatively little is known about residual neural networks (ResNets) in the nonlinear PDE setting. We prove that ResNets overcome the curse of dimensionality in the numerical approximation of solutions of semilinear heat equations with globally Lipschitz continuous, gradient-independent nonlinearities: under polynomial growth and network approximability hypotheses on the PDE data, there exist $η\in(0,\infty)$ and ResNets $Ψ_{d,\varepsilon}$, $d\in\mathbb{N}$, $\varepsilon\in(0,1]$, with at most $ηd^η\varepsilon^{-η}$ parameters whose realizations approximate the solution in dimension $d$ with an $L^2$-error of at most $\varepsilon$. The proof represents one deterministic realization of a multilevel Picard estimator by a ResNet whose shortcut connections transmit the spatial variable and a scalar accumulator, while the residual branches successively add the summands of the estimator. For ridge-sum initial conditions, admissible sigmoidal activations, and globally Lipschitz truncations of the nonlinearity, we obtain, for every $ξ>0$, the explicit bound $C_ξd^{4+ξ}\varepsilon^{-(3+ξ)}$ on the number of parameters.

0.37

Low Residuals, Wrong Value: Learning the Viscosity Solution of Hamilton-Jacobi-Isaacs Equations with PINNs

cs.LG math.NAmath.APmath.OC ·2026-10-19 ·Raul Jimenez, Pablo Tejerina-Pérez, Leonid Sarieddine, Pedro Tarancón-Álvarez, Marisol Traforetti +4 more
similarity 0.59
Abstract · 43 pages, 9 figures

Physics-informed neural networks (PINNs) and other residual-trained solvers judge a candidate solution by the size of its sampled PDE residual. For first-order Hamilton-Jacobi equations this is not sufficient: the same equation and boundary data admit Lipschitz almost-everywhere solutions that differ from the viscosity solution by $O(1)$ while violating the equation only on sets of measure zero. We study whether residual minimization selects such solutions in a fixed-horizon pursuit-evasion game whose Hamilton-Jacobi-Isaacs value function is known in closed form on the circle $S^1$ and on the flat tori $\mathbb{T}^{2}$ and $\mathbb{T}^{4}$. We train four neural parameterizations with ten seeds on each geometry, 120 runs in total. A standard network with periodic Fourier features converges to folded almost-everywhere solutions in all 30 runs: small residuals and the expected capture-layer gradient, yet values that are wrong over most of the domain. A network with periodicity imposed through the loss recovers the viscosity solution in one and two dimensions but becomes unreliable in four. Two parameterizations that receive the geodesic distance do substantially better: a single network fed the wrapped displacement and its distance recovers the viscosity solution in all 30 runs, while a minimum over periodic image charts succeeds in 27 of 30 and introduces a wrong-image failure mode. Finally, for the convex Hamiltonian considered here, we use the semiconcavity of the viscosity solution to build a mesh-free finite-difference test that needs only forward network evaluations; it detects the folded solutions and the harmful min-of-charts failures without access to the exact value function. A small sampled residual thus does not by itself validate a learned Hamilton-Jacobi solution; the parameterization and independent structural diagnostics can be essential for selecting the viscosity solution.

0.25

FOSLS-deRhaNN: native de Rham neural classes for H(div) and H(curl) with applications to first-order system least-squares neural network methods for partial differential equations

cs.LG math.NA ·2026-10-06 ·Shun Zhang
similarity 0.58
Abstract

We construct neural approximation classes native to the graph spaces H(div) and H(curl), in two and three dimensions and, for H(div), in any dimension. Every realization lies in the space for all parameter values, and with kinked potentials, such as ReLU networks, the admissible jumps appear at finite width. The classes are images of scalar and componentwise networks under fixed operators of the de Rham complex, and do not involve a mesh or finite element emulation. For H(div) in R^n two native classes are given on an equal footing, with a skew-symmetric potential $A$: $\mathrm{Div}\,A+R_nq+\mathbf{h}$, with the divergence $q$ as an explicit unknown, and $\mathrm{Div}\,A+\mathbf{z}$ with an $H^1$ field $\mathbf{z}$; for H(curl) the analogous classes are $\mathrm{grad}\,φ+Sr+\mathbf{h}$ in two dimensions and $\mathrm{grad}\,φ+\mathbf{z}$ in two and three dimensions. In all of them every interface jump of the field is carried by the potential term, $\mathrm{Div}\,A$ or $\mathrm{grad}\,φ$, while the remaining part has no interface jump (it is an $H^1$ field in the regular-decomposition classes); the classes with $\mathbf{z}$ are the componentwise approach enriched by this term. Known or learned interface geometry enters the potential through factors with trainable amplitudes, and the remaining part if the divergence jumps. The classes lead to the FOSLS-deRhaNN method, first-order system least squares with de Rham neural networks, whose loss is the least-squares functional posed in the natural spaces of the weak formulation; for elliptic equations this includes $H^{-1}$ right-hand sides and $H^{1/2}$ Dirichlet data. Elliptic equations with discontinuous coefficients and curl-curl problems are treated as instances, with the functional equivalent to the error; linear transport with discontinuous solutions and conservation laws with shocks use the same flux classes.

0.46

Distance-Residual Physics-Informed Neural Networks: A Deep Learning Framework for Differential and Partial Differential Inclusions

cs.NE math.NA ·2026-09-25 ·Maria Filipkovska, Juan José Marín, Isil Oner, Francisco Periago, Krzysztof Rykaczewski
theory similarity 0.58
Abstract

We introduce Distance-Residual Physics-Informed Neural Networks (DR-PINNs), a physics-informed learning framework for approximating solutions of ordinary and partial differential inclusions (DIs), governing laws in which a differential operator is constrained to lie in a set-valued map rather than equaling a prescribed function. The method replaces the classical pointwise PDE/ODE residual by the squared distance from the differential operator to the admissible set. This distance vanishes exactly when the inclusion is satisfied and measures the infimal correction needed for the operator to enter the admissible set. For a fixed closed convex admissible set, the squared distance is differentiable with respect to the operator value, with gradient given by the metric projection. When the admissible set also depends on the network state, that dependence is included through the chain rule. The framework encompasses ordinary and partial DIs with set-valued reaction terms. For both settings we prove consistency: under the stated closedness, measurability, convexity, and growth assumptions, any sequence of candidates satisfying the initial (and, in the parabolic case, boundary) conditions whose continuous distance-residual functional tends to zero admits a subsequence converging to an exact solution of the target inclusion. These are conditional statements for the continuous distance-residual functional; they do not cover the finite-collocation training loss, the behavior of the optimizer, or convergence rates. For admissible sets given as convex hulls of finitely many vertices, projection onto the set reduces to a small convex quadratic program, making the loss efficiently computable inside the training loop. Numerical experiments demonstrate high accuracy on the differential-inclusion benchmarks.

0.30

High Probability Derivative Bounds for Random tanh Neural Networks on a Hypercube

cs.LG math.NA ·2026-08-27 ·Josef Dick, Michael Feischl, Fabian Zehetgruber
theory similarity 0.57
Abstract · 31 pages, 0 figures

We establish high-probability bounds for mixed input derivatives of wide random neural networks whose activation derivatives satisfy a factorial growth bound. Our main result specializes these estimates to $\tanh$ networks with Xavier initialization. A direct deterministic analysis based on Euclidean operator norms of the weight matrices yields derivative bounds that generally grow exponentially with the depth. We show that this growth can be substantially improved for sufficiently wide Gaussian networks by isolating the term that is linear in the highest-order derivative and controlling the corresponding tangent directions by measurable finite nets. For scalar-output $\tanh$ networks with Gaussian weights and Xavier initialization, we prove that there exist constants $C,C_0,C_1>0$ such that, whenever the common hidden width satisfies $n \geq C\left(L^3n_0^2(1+\log n_0)+L^2\left(1+\log(L/η)\right)\right)$, then, with probability at least $1-η$, the estimate $\left|D^u\mathcal{R}_{Φ^{(L)}}(x)\right| \leq C_0 |u|! (C_1L)^{|u|-1}\prod_{j\in u}β_j(η,n_0)$ holds simultaneously for every non-empty $u\subseteq[n_0]$ and every $x\in[0,1]^{n_0}$. Thus, the first-order derivative bound is independent of the depth, while a square-free mixed derivative of order $|u|$ grows at most polynomially as $L^{|u|-1}$, apart from the coordinate factors. As consequences, we obtain high-probability bounds for the Euclidean Lipschitz constant and for weighted Sobolev norms of the network realization. The latter connect the derivative estimates to quasi-Monte Carlo integration and indicate how such regularity can enter the analysis of QMC-based training.

0.46

Non-Euclidean Gradient Descent Has Many Edges of Stability

cs.LG math.OC ·2026-10-10 ·Peter Zaika, Zakhar Shumaylov
theory similarity 0.57
Abstract

Recent observations that deep learning models train at the edge of stability have renewed interest in understanding stability properties of classical optimization algorithms. For quadratic objectives, standard gradient descent has a sharp stability threshold: it converges from every initialization when $η<2/L$, diverges from almost every initialization when $η>2/L$, and diverges from every nonzero initialization when $η>2/μ$. For non-Euclidean gradient descent, the corresponding known guarantees are weaker: convergence holds for every initialization when $η<2/S$, where $S$ is the generalized sharpness, while every nonzero initialization diverges when $η>2/μ$. The behavior in the intermediate regime $2/S<η<2/μ$ is not fully characterized. We show that Hilbertian norms are the only ones for which $2/S$ is an almost-everywhere sharp threshold for every positive-definite quadratic objective. Using linear conjugacy between non-Euclidean and Euclidean gradient descent, we show that for general norms the intermediate regime can contain arbitrarily many edges of stability associated with distinct attracting directions. We identify a forward-invariant region of exponentially divergent directions beyond the generalized stability threshold. When the directions of minimum curvature are finite and nondegenerate, we prove that, sufficiently close to $2/μ$ from below, only their invariant directions remain convergent. We give smooth two-dimensional constructions with arbitrarily many distinct thresholds, arbitrarily large relative threshold separation and show these properties to be stable.

0.12

Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks

cs.LG stat.ML ·2026-09-17 ·Xianjun Li, Yunfei Yang
similarity 0.57
Abstract

Recent studies have shown that smooth functions can be well approximated by ReLU neural networks with path norm constraint on the weights. We extend these results from uniform approximation to approximation in Sobolev norm. Specifically, we analyze how well Sobolev functions in $W^{n,p}$ can be approximated by neural networks with width $W$, depth $L$ and path norm bounded by $K$, when the approximation error is measured in the $W^{1,p}$-norm. For shallow networks with depth $L=1$, we derive the approximation error bound $\mathcal{O}(\max\{W^{-(n-1)/d}, K^{-(n-1)/(s-n)}\})$, when the smoothness index satisfies $n<s=(d+3)/2$ and the input is $d$-dimensional. For deep networks, we remove the restriction on the smoothness by showing that the approximation bound $\mathcal{O}(K^{-(n-1)/(d+d/p+1)})$ holds if the width $W$ and depth $L$ are sufficiently large.

0.07

Variationally correct physics-informed operator learning for nonlinear partial differential equations

cs.LG math.NA ·2026-11-16 ·Yuan Qiu, Wolfgang Dahmen, Peng Chen
theory similarity 0.57
Abstract

Physics-informed operator learning trains neural operators on the residual of a parametric partial differential equation (PDE) instead of labeled solution data, but its reliability depends on a variationally correct residual: one that is equivalent to the error in a norm dictated by the equation, the PDE-compliant norm. For nonlinear equations written as first-order systems, we prove that the first-order system least-squares residual is equivalent to the error in this norm near a solution branch if the linearized residual is stable and the residual derivative is Lipschitz continuous, possibly in a norm stronger than the error norm. We verify these conditions for monotone reaction, solution- and gradient-dependent diffusion, and compressible Neo-Hookean hyperelasticity. The stability mechanisms vary across these classes, as do the regions where the equivalence holds, from arbitrary bounded sets to a neighborhood of the branch. In a conforming finite element space, we show that the loss excess of a state near the finite element solution is equivalent to its squared distance from that solution. We use this loss to train a reduced basis neural operator with values in this space, with solution snapshots needed only for the basis and the loss and its symbolically derived gradient assembled by the finite element library. At inference, this loss estimates the error. Numerically, the square root of the loss excess tracks the error within factors comparable to those of the linear case, and the operator achieves higher accuracy in the PDE-compliant norm than grid-based baselines evaluated in the same finite element space.

0.22