CS Paper Radar

On the gradient conjecture at infinity: Finite spherical length for gradient trajectories of definable potentials

Thang Do, Arnulf Jentzen, Benno Kuckuck

cs.LG math.DSmath.AGmath.CAmath.LO ·submitted 2026-10-21 05:06 UTC · v2
theoryunusual

Abstract

We prove that for every open $U \subseteq \mathbb{R}^n$ the spherical projection of every gradient trajectory $θ(t)$, $t \geq 0$, of a definable $C^1$-potential $F \colon U \to \mathbb{R}$ has finite length and, in particular, converges (directional convergence). In other words, every gradient trajectory of a definable $C^1$-potential has finite total angular motion. Here, definable means definable in an arbitrary o-minimal structure expanding the real field. Notably, the gradient trajectory in this result is an arbitrary solution of the gradient flow differential equation (whose solutions may not be unique) with no further assumptions: The gradient trajectory may be unbounded and diverge to infinity or it may be convergent. The established theorem therefore provides affirmative answers to both the gradient conjecture at infinity for definable potentials and the definable Thom gradient conjecture. We extend our conclusions from the Euclidean metric to a general class of ambient metrics including Riemannian metrics and we establish uniform local upper bounds for the lengths of the spherical projections of gradient flow trajectories. We finally apply our findings to the training of deep neural networks (DNNs) to conclude that, in the case of definable and continuously differentiable activation and loss functions, the spherical projection of the DNN training flow must always converge (directional convergence).

Comment
143 pages

Closest papers

Nearest neighbours by meaning across the whole index.

RW-Flow: One-Step Generation on Compact Manifolds via Wasserstein Gradient Flows

cs.LG ·2026-09-30 ·Ualibyek Nurgulan, Seungwoo Yoo, Prin Phunyaphibarn, Minhyuk Sung
hot direction similarity 0.53
Abstract · 27 pages

Manifold-valued data, and consequently the distributions they induce, are prevalent across many domains, ranging from the locations of geospatial events, such as earthquakes, to biomolecular torsion angles that encode information about three-dimensional structure. While diffusion and flow-based generative models have been successfully extended to compact manifolds, sampling typically requires tens or hundreds of sequential network evaluations. We introduce RW-Flow, a theoretically grounded framework for learning one-step generative models on compact manifolds via Wasserstein gradient flows. The main challenge is identifiability: driving the velocity field to zero should guarantee that the model distribution matches the target distribution. We establish a necessary and sufficient condition for identifiability on compact, connected Riemannian manifolds. We specifically show that, for a symmetric, Lipschitz-continuous cost function, the velocity field induced by the Sinkhorn divergence is identifiable if and only if the associated Gibbs kernel is nondegenerate. This characterization provides a general principle for designing identifiable costs on compact manifolds. It also reveals that the squared geodesic distance, the natural manifold analogue of the squared Euclidean distance, does not always guarantee identifiability. Across benchmarks involving geospatial events, protein side chain torsion angles, RNA backbone torsion angles, and general manifolds discretized as triangular meshes, RW-Flow outperforms existing one-step methods in nearly all settings under fair comparison conditions.

0.25

Riemannian ascent--descent for nonconvex nonconcave minimax landscapes: convergence to basin saddle points and applications to distributionally robust optimization

cs.LG math.OCstat.ML ·2026-09-12 ·Rishabh Dixit, Pranav Upadrashta, Alex Cloninger
hot direction similarity 0.50
Abstract

We study a class of distributionally robust optimization (DRO) problems for the statistical risk problem, formulated as minimax problems over the product of a Euclidean space and a Riemannian manifold. Because the resulting minimax landscape is nonconvex nonconcave in general, no globally convergent first order method is known to be available. We instead introduce the notion of a \emph{basin saddle point}, a Nash equilibrium defined locally on the Cartesian product of a $δ$ basin around a connected component of the local minima critical set and a geodesic ball on the measure manifold. We develop an abstract convergence framework for a Riemannian gradient ascent multistep descent iteration to a basin saddle point under a local Łojasiewicz type growth condition, with exponent $β\in (1,2]$, in the $δ$ basin around connected components of the local minima critical sets. Under Lipschitz regularity of critical sets we establish linear convergence for $β= 2$ and polynomial convergence for $β\in (1,2)$ to a basin saddle point, with explicit dependence on the sectional curvature of the manifold. We then instantiate this framework for the statistical risk DRO problem over Gaussian measures, where the ambiguity set is naturally modeled as the product of Euclidean space and the Bures Wasserstein manifold of covariance matrices, which we relax to a penalized DRO formulation. We derive nonasymptotic Hessian estimates for the resulting Lagrangian, establish existence and local uniqueness of its maximizer, and prove that an alternating Riemannian gradient scheme converges to a basin saddle point of the penalized DRO problem, recovering the linear and polynomial rates of the abstract theory with all constants explicit in terms of data dimension, loss moments, and the reference covariance.

0.42

Incremental Stability of Gradient Flows under Generalized Polyak-Łojasiewicz Inequalities and Its Implications for General Systems

eess.SY math.OC ·2026-10-29 ·Andreas Oliveira, Arthur C. B. de Oliveira, Mario Sznaier, Eduardo Sontag
similarity 0.50
Abstract

Gradient flows of twice continuously differentiable strongly convex and convex objectives are, respectively, incrementally exponentially stable and non-expansive in the Euclidean metric. In this work, we investigate to what extent these properties extend to nonconvex objectives satisfying generalized Polyak-Łojasiewicz inequalities. For objectives with a unique minimizer, we establish conditions under which the gradient flow is contracting in a state-dependent Riemannian metric and incrementally exponentially stable. For objectives with nonunique minimizers, we generalize the non-expansivity of convex objectives to incremental stability for more general nonconvex objectives. We focus on incremental stability due to its implication to convergence to equilibrium points; when paired with a Lyapunov function it implies pointwise convergence to a minimizer and continuity of the endpoint map. Along the way, we derive new structural results for incrementally stable nonlinear systems and extend the analysis to Heavy-Ball and Dynamical Inertial Newton flows.

0.28

Kähler landscapes for complex neural network descents and guarantees including a search and destroy of the Calabi-Yau manifold

cs.LG math.DGstat.ML ·2026-08-20 ·Andrew Gracyk
theory similarity 0.49
Abstract

We study landscapes for complex-parameterized networks. Our approach is motivated with an information-theoretic manifold perspective of the parameter and via classical optimization guarantees although of complex geometric variety such as through Dolbeault asymptotics. The descent path admits a Kähler information metric under a cross-entropy via the Wirtinger Hessian on the log-likelihood potential. We restrict attention to a descent update rule with natural gradient descent via a differentiated loss scaled by the inverse metric, so the descent path remains in the holomorphic tangent bundle. We emphasize Calabi-Yau information manifolds which profane theoretical guarantees via an ill-curvature-conditioned, or lack thereof, landscape. We focus on Calabi-Yau metrics specifically in a non-compact setting with a global potential, so defined geometrically rather than invoking the topological requirements of the Calabi conjecture. In non-compact settings, we can write the metric determinant with respect to a background in terms of a function, pluriharmonic or real. Using Yau's Liouville theorem, we get a partial eigenvalue blow-up effect. In an empirical setting, a Ricci-flat metric will not form, but a blow-up can be a local condition and can hold empirically on open sets even partially in generalized Kähler cases due to lower bounds on the Ricci potential. Moreover, it has been discovered that negative curvature subverts the loss landscape, specifically sectional curvature, so we expand on this and draw interconnections to negative-definite Ricci curvature. Our arguments primarily exist via geometric modality, although we establish roots in deep learning theory such as through asymptotics at initialization and connections through failure modes of neural network guarantees under vanishing and negative Ricci curvature.

0.26

On the Convergence of Continuous-Time AdamW

cs.LG math.OC ·2026-11-16 ·Bangjie Wang, Ning Ning
similarity 0.49
Abstract · 55 pages, 9 figures

Continuous-time AdamW provides a framework for understanding whether an optimizer widely used to train large-scale neural networks settles to an equilibrium or sustains persistent motion. We study this long-time behavior while retaining bias correction and fixed positive weight decay. Under suitable assumptions, we establish that the dynamics are well defined for all time and remain bounded when the decay is sufficiently small. Our main result connects their limiting behavior to the geometry of the objective: trajectories approach a locally invariant manifold whose dimension equals the Hessian nullity along the associated critical component. When critical points are isolated and have at most one such direction, the trajectory converges to a single equilibrium. With two degenerate directions, however, bounded persistent motion can occur even for a smooth, strictly convex objective with a unique minimum. We also quantify how close the long-time trajectory remains to the critical set as weight decay decreases, without assuming convergence to a point. These results identify objective geometry as a key to understanding when continuous-time AdamW converges and when persistent motion is possible.

0.08

Compatible Likelihoods for Flow Matching on Manifolds

cs.LG stat.ML ·2026-10-22 ·Lucas Ng, Georgios Batzolis, Mark Girolami
published similarity 0.49
Abstract · 40th Conference on Neural Information Processing Systems (NeurIPS 2026)

On the simplest non-Euclidean manifold, the 1D flat torus, we exhibit a distribution where a recently proposed extension of Riemannian flow matching is globally optimized by a velocity field that pushes samples away from the data distribution. We trace this failure mode to a property we call compatibility; a likelihood-based flow matching objective is compatible if its population minimizer recovers the marginal velocity field induced by the data and coupling. We give a general condition for compatibility, apply it to existing methods, and bound the worst-case error in terms of a residual component vanishing under compatibility. Next, we note that existing compatible objectives put density on velocities that are infeasible on compact manifolds. We propose an objective built from an exponential family over endpoints, with the conditional velocity as sufficient statistic, which is both compatible and supported only on feasible velocities. This manifests as a reweighting of the flow matching loss which, on constant-curvature manifolds, treats errors in flow speed and direction differently. Empirically, our method recovers the correct velocity field on the test case, outperforms prior methods on high-dimensional product tori, and obtains substantial NLL improvements over prior work on two of four standard geospatial benchmarks on the sphere at moderate runtime overhead. On a large dataset of high-dimensional human poses, our method outperforms prior work according to sample-based metrics.

0.25

Trajectory Manifolds for Nonlinear Data-Enabled Predictive Control

eess.SY math.OC ·2026-09-16 ·Arda Bayer
theory similarity 0.49
Abstract · 15 pages

This note establishes a geometric foundation for trajectory-manifold representations of deterministic nonlinear systems in a behavioral setting motivated by data-enabled predictive control. For a discrete-time system $x_{k+1}=f(x_k,u_k)$ with measured state and a $C^r$ transition map, $r\geq 1$, we consider the terminal-state-augmented finite-horizon behavior consisting of all admissible state-input trajectories over a prediction horizon $N$. We prove that this behavior is a $C^r$ embedded submanifold of the ambient trajectory space with intrinsic dimension $n+Nm$, where $n$ and $m$ are the state and input dimensions. Moreover, the rollout map from the admissible initial-state and input coordinates $(x_0,\mathbf u)$ is a $C^r$ diffeomorphism onto the behavior manifold, providing explicit global smooth coordinates. This yields a canonical exact encoder--decoder representation and implies that any exact differentiable latent representation of the full behavior must have latent dimension at least $n+Nm$. The geometric result does not require controllability, stabilizability, or invertibility of the dynamics. Corresponding results are given for zero-order-hold sampled continuous-time systems and fixed-step numerical transition maps. These results provide the deterministic geometric foundation for subsequent data-driven approximation and predictive-control development.

0.49

Geodesic strong convexity does not imply forward invariance under gradient flow on SO(3): a certified counterexample

eess.SY ·2026-08-29 ·Dongming Wang, Wei Ren
similarity 0.49
Abstract

Let $mathcal{C}=\overline{\mathcal{B}}_ρ(R_c)$ be a geodesic ball of radius $ρ<π/2$ in SO(3) with the bi-invariant metric, and let $f$ be geodesically strongly convex on $\mathcal{C}$ with an interior minimizer. It is tempting to expect the gradient flow $\dot R=R(-\nabla f)^\wedge$ to keep $\mathcal{C}$ forward invariant: the flow is attracted to an interior point, and strong convexity appears to leave no room for outward motion. We show this expectation is false by an explicit, fully certified construction with $ρ=0.3$: a cost, quadratic in the principal logarithmic chart with off-diagonal coupling $0.7$, whose geodesic Hessian satisfies $\Hess f\succeqμI_3$ on all of $\mathcal{C}$ with a machine-certified modulus $μ\geq0.172$, rigorous ball arithmetic over exact rational inputs, yet whose descent velocity at a boundary point has the exact rational outward radial component $21/500$. A continuity corollary of the exact rate certifies that the flow exits the ball; numerical integration puts the peak excursion near $0.3143$ before convergence to the minimizer. The mechanism is elementary: strong convexity constrains the projection of the gradient onto the minimizer direction, not onto the inward radial direction. Code reproducing every certified constant and figure accompanies the note.

0.45