CS Paper Radar

Differentiable High-Order Mesh $r$-Adaptivity

Ketan Mittal, Julian Andrej, John Camier, Veselin Dobrev, Tzanio Kolev, Leonardo Molinari, Vladimir Tomov

cs.MS math.NA ·submitted 2026-11-17 02:33 UTC · v1
benchmark

Abstract

High-order mesh r-adaptivity typically entails minimizing a nonlinear functional using its gradients and, for Newton-based methods, its Hessian. These derivatives can be expensive to derive, must be updated with any changes to the functional, and errors in the derivatives directly impact the convergence of r-adaptivity. Additionally, derivatives of the r-adaptivity operator are required for consistent sensitivity analysis for inverse problems. This work develops a framework for differentiable r-adaptivity based on the Target-Matrix Optimization Paradigm. Given a pointwise functional defining the local mesh-adaptation objective, the proposed framework uses $\partial$FEM to automatically generate pointwise first- and second-order derivatives. These local derivatives are then assembled into a global gradient vector and a Hessian operator through the finite element operator decomposition. The derivative kernels support portable CPU and GPU execution, with partial assembly and tensor sum factorization enabling matrix-free application of the Hessian operator. The framework also provides consistent sensitivities of the r-adaptivity operator through automatically generated mixed derivatives and implicit differentiation of the mesh stationarity equations. Numerical experiments show GPU solve times comparable to those of hand-derived TMOP implementations on a benchmark problem, improvement in r-adaptivity convergence due to accurate computation of the derivatives, and demonstrate the use of mesh sensitivities for material-interface identification in an inverse heat-conduction problem.

Comment
14 pages, 10 figures, 2 tables

Closest papers

Nearest neighbours by meaning across the whole index.

JAXIGA - A software framework for GPU-enabled and differentiable isogeometric analysis

cs.MS math.NA ·2026-10-06 ·Cosmin Anitescu, Yongjie Jessica Zhang
benchmarkcode/dataunusual similarity 0.65
Abstract

In this paper, we present JAXIGA, an open-source library for isogeometric analysis based on JAX. The library separates the construction of the mesh from the computations which are traced for compilation and differentiation. This allows geometry evaluation, assembly and the supported solvers to be combined in a just-in-time compiled program, for which first-order reverse-mode derivatives can be computed. We consider three numerical methods obtained from a single pointwise energy density. The Galerkin and energy-minimization methods use the same discrete potential and have the same stationary points, while collocation enforces the corresponding strong form. The sensitivities of a converged solution are obtained by solving one adjoint problem, and their accuracy depends on the tolerances of both the primal and adjoint problems. We also describe hierarchical extraction for local refinement of the variational methods and the use of smooth spline spaces for fourth-order problems. The numerical results are compared with analytical solutions and independent FEniCSx and JAX-FEM computations. For the reported GPU benchmark with 36,448 unknowns, assembly is twenty-four times faster than on the host CPU. When the conjugate-gradient solver is also run on the GPU, the total solution time is reduced from 31.8 to 1.38 seconds, and the result agrees with the direct solution to $5\times10^{-11}$. These results show that the choice of solver and the reuse of compilation are important for obtaining an overall reduction in computational cost.

0.35

Primal Acceleration of Newton's Method

cs.AI math.OCcs.LG ·2026-08-21 ·Nikita Doikov
similarity 0.60
Abstract

We develop a new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian. The algorithm uses only primal variables and performs just one linear solve per iteration. With a simple predetermined choice of parameters, it achieves the global convergence rate of $O(1/k^3)$ in terms of the functional residual. To the best of our knowledge, this is the first second-order method for this problem class attaining this rate while relying solely on one linear system solve per iteration (without solving auxiliary nonlinear regularized subproblems, such as cubic regularization, performing nonlinear parameter searches, or using dual extragradient corrections). Our method can be implemented in a Hessian-free way, using an inexact linear system solver, while preserving the fast global rate. We further extend our construction to arbitrary geometry through Bregman divergence, and to composite optimization problems.

0.11

Axion: A differentiable, GPU-native finite element framework,demonstrated on phase-field fracture

cs.CE ·2026-11-18 ·Wei Wang, Meenu Krishnan U., Somdatta Goswami
benchmarkcode/data similarity 0.60
Abstract · 35 pages, 9 figures

We introduce Axion (Autodifferentiable, XLA-compiled, Implicit, On-device Numerics), a differentiable, GPU-native finite element framework written in JAX. The element kernels, Newton and Krylov iterations, bound-constrained updates, staggered coupling of fields, and the loop over load increments are expressed as a single program that is just-in-time compiled and executed on the GPU without host intervention. Consistent tangent operators are obtained by forward-mode automatic differentiation of the element residual, and derivatives of the discrete solution with respect to model parameters are obtained by differentiating the same program. Global operators are applied element by element without assembling a sparse matrix, octree mesh refinement is performed within fixed-size arrays, and a cell-based domain decomposition runs the same program on several GPUs. Phase-field fracture is implemented as the first module because it combines fine mesh resolution, bound-constrained damage, and repeated nonlinear solves over a loading history, which together make it computationally expensive. On two- and three-dimensional single-edge-notched tension benchmarks, a uniformly refined Axion computation completes faster than the corresponding adaptive FEniCSx computation. At identical discretization, the two-dimensional benchmark runs 37.9 times faster on one GPU than a FEniCSx implementation on one CPU. Strong and weak scaling efficiencies on four GPUs are 92% and 96%, respectively. Large-scale demonstrations include hydraulic fracture in a poroelastic medium (0.38M degrees of freedom), fracture of a multi-material premolar under contact loading (0.69M degrees of freedom), and a concrete gravity dam (96M degrees of freedom) on a single GPU. The implementation is available as open-source software.

0.48

lapanda: A Matrix-Free Differentiable Solver for Nonconvex Constrained Optimization Layers

cs.LG math.OCeess.SY ·2026-09-27 ·Yuankun Chen, Zifei Nie, Kangyu Lin, Ján Drgoňa, Liang Wu
similarity 0.58
Abstract

Differentiable optimization brings the structural guarantees of mathematical optimization to network pipelines, allowing them to be trained end-to-end. However, its application remains challenging for nonconvex constrained problems, as existing differentiable solvers often suffer from limited modeling expressiveness due to their reliance on specialized problem structures, while also incurring substantial computation time and memory overhead in both the forward and backward passes. To address these challenges, we propose lapanda, a matrix-free differentiable solver for nonconvex optimization with general constraints. It reformulates the problem to a sequence of augmented Lagrangian subproblems, each handled by a first-order inner solver through a proximal averaged quasi-Newton algorithm with adaptive linesearch, thus enabling efficient forward optimization. We establish local well-posedness of the solution map and convergence of the outer iterations, and further derive a sensitivity alignment between the original problem and the final subproblem in the backward pass, demonstrating that the subproblem sensitivity, which can be computed efficiently in a matrix-free manner, provides a principled approximation to the exact optimizer sensitivity. We evaluate lapanda on nonconvex constrained Rosenbrock benchmarks, imitation learning with several representative constrained optimal control problems, and embedded robotic obstacle-avoidance tasks. Compared with state-of-the-art differentiable solvers, lapanda delivers substantial reductions in computation time and memory footprint while maintaining reliable constraint satisfaction and learning performance.

0.18

Benchmarking Optimizers to Solve Inverse Problems with Differentiable Physics Simulators

cs.LG math.OCphysics.comp-ph ·2026-09-12 ·Xiang Chen, Huanhuan Xia
similarity 0.56
Abstract

Solving inverse problems with differentiable physics simulators holds the potential to revolutionize scientific discovery and engineering design, as it enjoys both the strict physical correctness from rigorous numerical physics simulators, and the high efficiency and effectiveness from automatic differentiation and gradient-based optimization. However, currently, this paradigm faces performance issues in optimization. In this work, we target benchmarking the performance of different optimizers to solve various inverse problems. We construct 12 differentiable physics simulators spanning physics domains including discrete mechanics, continuous mechanics, atomistic simulations, rendering, and semi-empirical physics models. Based on these simulators, we design corresponding inverse problems that can be categorized into parameter identification, inverse design, and optimal control. Finally, we conduct extensive experiments to compare the performance of different optimizers, including regular first-order methods, approximate second-order methods, as well as global optimizers, on these inverse problems, and analyze the results to provide insights on how to choose and design optimizers for differentiable programming. We hope such benchmarks can inspire the development of more effective optimizers, and further promote the applications of differentiable programming in various scientific and engineering domains.

0.22

Low-rank corrections to a shared reference Jacobian for GPU-parallel AC $N\!-\!1$ contingency analysis

eess.SY cs.DCmath.NA ·2026-11-02 ·Marc Hunkemöller, Nico Westerbeck, Joost van Dijk, Lars Schewe, Dirk Witthaut
similarity 0.56
Abstract · Submitted to IEEE Transactions on Power Systems. 10 pages, 5 figures, 1 table

Contingency analysis repeatedly solves the power-flow equations under many single-element outages, an increasingly demanding workload as operating margins tighten and renewable variability grows. Graphics processing units (GPUs) are attractive for this task, but a change in topology alters the sparsity pattern of the Jacobian, preventing the uniform batched factorization on which efficient GPU solvers rely. This paper presents a GPU-parallel Newton--Raphson solver for $N\!-\!1$ contingency analysis that separates the change in operating state from the change in topology. A single reference Jacobian is inverted once and reused across all contingencies. Each topology change is expressed as a low-rank Woodbury correction, while the residual state-dependent perturbation is handled by an inner iterative solve, a truncated Neumann series or the biconjugate gradient stabilized method. We analyze the spectral radius and condition number of the resulting iteration matrices and show that the reference Jacobian is an effective preconditioner. On transmission-scale benchmarks, the method attains per-topology solve times competitive with an optimized 80-core central processing unit baseline, with the low-rank correction accounting for approximately 1--2\% of the workload. This makes GPU-parallel $N\!-\!1$ contingency analysis feasible at transmission scale without per-topology refactorization.

0.33

Joint Structure Identification and Newton Acceleration via Proximal Line Search for Nonconvex Optimization

eess.SP ·2026-09-16 ·Yiming Zhou, Wei Dai
theory similarity 0.55
Abstract

We consider composite optimization with a smooth, possibly nonconvex function and a separable convex polyhedral term. Such problems have a simple nonsmooth structure but arise widely in applications. Existing Newton-type methods may fail to identify nonsmooth coordinates during direction computation, move coordinates away from those reached by the full step during line search, or rely on separate first-order steps for identification. In this work, we propose a unified inexact proximal Newton method that couples structure identification with second-order acceleration through a proximal line search designed to retain the nonsmooth structure revealed at the local model. In particular, on a predicted working set, we solve a quadratic model inexactly but enforce its constraints exactly, allowing coordinates to reach new kinks without leaving the selected affine intervals. This set also includes small-margin kink coordinates to exploit the Hessian coupling effect. To globalize the direction, we construct a proximal line search using an isotropic proximal model whose linear term is chosen so that the initial trial exactly recovers the full step. Backtracking increases the proximal curvature until sufficient decrease holds, allowing coordinates to remain at kinks reached by the full step while damping other components. Generating each trial point requires only proximal operator evaluations. We prove finite backtracking and finite active-set identification under strict complementarity and a non-singular reduced Hessian, together with local Q-superlinear convergence under a directional Dennis-More condition and vanishing relative inexactness. Numerical results verify the effectiveness of the proposed method.

0.12

Newton's Method for CP Decomposition

cs.MS ·2026-11-16 ·DuBose Tuller, Grey Ballard, Hussam Al Daas, Nicholas I. M. Gould
similarity 0.55
Abstract · 30 pages, 7 figures

We present an algorithm for computing the CP tensor decomposition using Newton's method. The problem of fitting a CP model to a data tensor with least squares loss can be solved by various gradient-based optimization techniques, including Gauss-Newton methods that exploit the tensor structure of the problem and iteratively solve the linear system involving the approximate Hessian matrix to determine the step direction at each iteration. Gauss-Newton methods exhibit superlinear convergence when the residual is small but can otherwise struggle to converge when the approximation to the Hessian is not accurate. We show that the Gauss-Newton approach can be extended to Newton's method, using the true Hessian to obtain fast convergence even in the case of large residuals. In particular, we derive efficient means of applying the true Hessian to vectors for use within an iterative solver, we employ globalization and regularization techniques to make the Newton model robust to ill-conditioned and/or indefinite Hessians, and we propose hybrid use of Gauss-Newton and Newton steps to balance convergence behavior both far and near local minima. Our numerical experiments use synthetic examples to show in which scenarios Gauss-Newton and other methods struggle to converge, and we demonstrate the superiority of the hybrid approach on two application data sets.

0.35