CS Paper Radar

Mean-Field Oscillator Ising Machines: Gradient Flows and Classification of Limit Solutions

Arvind R. Venkatakrishnan, Max Emerick, Bassam Bamieh, Francesco Bullo

eess.SY math.OCmath.DS ·submitted 2026-08-17 02:36 UTC · v2

Abstract

Oscillator Ising Machines (OIMs) have emerged as promising computational architectures for approximating solutions to combinatorial optimization problems. We derive and analyze the mean-field limit of an OIM model and show that it inherits the gradient flow structure of the finite-dimensional dynamics. We identify conditions under which this mean-field evolution admits an Eulerian formulation as a gradient flow on the Wasserstein space of probability measures and contrast this with a Lagrangian formulation which is always available. This gradient flow structure strongly constrains the long-time dynamics and enables a complete classification of limit solutions and their stability in the totally symmetric case. In particular, all limit solutions are fixed points whose phases cluster into at most four groups, and for almost all parameter values, only binarized fixed points -- those with clusters at $0$ and/or $π$ -- can be stable. Since binarized states are exactly those for which a feasible solution to the original problem can be read out, this shows that feasible solutions can almost always be recovered. We provide tight bounds on the parameter thresholds for which fixed points in this binarized family are stable, thereby identifying the threshold for binarization in this model, and present numerical evidence that the mean-field model correctly predicts behavioral regimes in certain large random networks, including Erdős-Rényi networks. Moreover, this problem provides a case study in the application of mean-field models, gradient flows, and optimal transport to the study of large-scale systems, and we highlight a number of features of this problem which are informative in this respect.

Comment
31 pages, 5 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Basin-Preserving Discretizations of Modern Hopfield Retrieval Dynamics: Energy Cells, Dissipation, and the Attention Limit

cs.NE math.NA ·2026-08-21 ·Francisco R. Villatoro
theory similarity 0.58
Abstract · 37 pages, 1 figure, 4 tables. Reproducibility code and numerical data: Zenodo DOI 10.5281/zenodo.22048179

The retrieval dynamics of a modern Hopfield network is the gradient flow of a log-sum-exp energy, while the attention update is its exact difference-of-convex minimization step. We study which time discretizations preserve not only energy decay and equilibria but also basins of attraction. We introduce energy cells, connected components of sublevel sets containing one attractor and no other critical point. Our main theorem shows that every finite energy cell below the escape energy is contained simultaneously in the basin of the continuous flow, every relaxed attention map $Ψ_θ=(1-θ)\,\mathrm{id}+θ\,\mathrm{attention}$ for $0<θ<2$, and implicit Euler throughout its uniqueness regime. A parameter-uniform unit-curvature majorant yields unconditional dissipation and a monotone interpolation of each discrete step. We also derive explicit local contraction bounds near well-separated patterns, with a certified optimal slight overrelaxation; characterize proximal tunneling and overshoot beyond the preservation regimes; compare first-order error constants; establish an order barrier for scalar reparametrizations of the relaxed family; construct a second-order scalar-auxiliary-variable scheme; and extend cell preservation to damped difference-of-convex iterations in Bregman geometry, including a certified overrelaxed window under bounded asymmetry. Nine numerical campaigns test the bounds and failure mechanisms. In two-dimensional basin experiments, all observed disagreements between continuous and discrete retrieval occur above the attractor-specific numerically inferred escape level.

0.32

Deep Learning Theory through the Lens of Statistical Physics: A Topical Review

cs.LG cond-mat.stat-mech ·2026-11-06 ·Wei Huang, Youjin Deng
publishedsurvey similarity 0.57
Abstract · 22 pages, 5 figures, 1 table. Topical review published in Front. Phys. 22(1), 011302 (2027); this version contains minor corrections

The recent success of foundation models and deep learning has far outpaced our theoretical understanding of how large-scale neural networks learn and generalize. Statistical physics provides a natural language for describing such high-dimensional, nonlinear, and strongly coupled systems, offering concepts such as mean-field theory, phase transitions, energy landscapes, and stochastic dynamics. In this topical review, we adopt a statistical-physics way of thinking -- from concepts including phases and criticality, and free-energy viewpoints, to methods such as kernel and Gaussian-process limits, mean-field and dynamical mean-field theories, random-matrix tools, and Fokker-Planck formalisms, and to a mindset emphasizing universality, scaling, and tractable limits with explicit validity regimes. We survey recent progress on understanding modern deep learning through this lens, including mean-field analyses of wide networks, spectral and geometric perspectives on loss landscapes, stochastic-gradient-based training viewed as a dynamical process, and phase-transition analogies that organize phenomena such as overparameterization, generalization, and representation change. Our goal is to synthesize these threads into a coherent framework that clarifies what each approximation explains, when it is expected to hold, and how apparently competing narratives relate or diverge. Together, these perspectives form a unified physical framework that views representation learning as a collective phenomenon in high-dimensional systems, bridging deterministic and stochastic regimes of gradient-based optimization.

0.24

Optimal Mixing of Glauber Dynamics for the Sherrington-Kirkpatrick Model at $β< 1/2$

cs.DS math.PRmath-ph ·2026-08-23 ·Sihan Wang
theoryhighly cited similarity 0.55
Abstract · Revised version; added related references and improved exposition

We prove that for every fixed inverse temperature $β< 1 / 2$, with high probability over the disorder, the single-site Glauber dynamics for the $n$-spin Sherrington-Kirkpatrick model mixes from every initial configuration to within total variation distance $\varepsilon$ in $O_β\left(n \log\left(n / \varepsilon\right)\right)$ steps. The bound holds uniformly over all external fields and is optimal up to constants depending only on $β$. The main ingredient is a deterministic criterion for optimal-order Poincaré inequalities in general Ising models, established via the integrated Bakry-Émery criterion together with a new two-spin estimate. A standard application of the localization-scheme framework of Chen and Eldan then upgrades the Poincaré inequality to a modified log-Sobolev inequality, yielding the optimal mixing-time bound. The main ideas underlying the proof of the Poincaré inequality were generated by GPT-5.6 Sol Ultra.

0.46

Finite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows

cs.LG stat.ML ·2026-10-07 ·Ryotaro Kawata, Atsushi Nitanda, Taiji Suzuki
theory similarity 0.55
Abstract

Wasserstein gradient flow extends gradient descent to probability measures. Its Hessian-guided perturbed variant (PWGF) adds Gaussian perturbations to escape saddle points in nonconvex problems. We investigate when its approximation by finitely many interacting particles remains accurate over growing time horizons. Our analysis retains the curvature accumulated along the population-driven reference path: negative curvature can amplify approximation errors, while subsequent positive curvature can damp their influence. This captures favorable scenarios in which temporary instability is compatible with accurate tracking over growing horizons. Under regularity assumptions and a prescribed common perturbation schedule, we prove particle and objective-value tracking bounds on a high-probability event for reference paths satisfying explicit conditions on accumulated curvature. To handle state-dependent Gaussian jumps, we construct a population-first coupling that preserves the reference particles' conditional independence and reduces jump errors to covariance comparison. We verify the conditions in a variance-plus-cosine model, where curvature recovery yields a growing-horizon tracking guarantee. We also establish local attraction, transverse descent, and positive second variation in two regions of a regularized matrix-factorization model, motivating a positive-negative-positive curvature pattern.

0.16

Infinite ResNets are Gaussian Mean ODEs

cs.LG ·2026-10-30 ·Louis-Pierre Chaintron, Lénaïc Chizat, Javier Maass
similarity 0.55
Abstract

Understanding the training dynamics of deep neural networks in large-scale limits is a question both of foundational and of practical importance. Prior works have obtained such descriptions through dynamical mean field theory (DMFT), yielding a system of non-Markovian dynamics that are challenging to analyze and that obscure the optimization structure of the original dynamics. In this paper, we obtain instead a simple description of the large-scale limit training dynamics for ResNets in standard shape scalings and in the $1/\text{depth}$ branch scaling that allows maximal (local) feature updates. For a model trained with gradient descent (GD) from a random initialization, the limit dynamics is simply GD on a Neural Mean ODE with Gaussian initial weights. Our analysis covers population or empirical risk GD in the joint limit of large width, depth and embedding, input, and output dimensions. We obtain approximation scaling laws that depend on the shape of the ResNet and the covariance spectrum of the data. We also illustrate the flexibility of our approach by discussing Mixture of Expert models, where likewise the limit admits an intuitive (nested) Mean ODE description.

0.20

Stochastic Gradient Descent over P2

cs.LG stat.MLmath.PR ·2026-09-11 ·Maria Oprea, Qin Li, Yunan Yang
similarity 0.54
Abstract

Stochastic gradient descent (SGD) admits diffusion approximations that replace the complicated randomness of stochastic gradients by Gaussian noise, providing a powerful tool for understanding its dynamics and long-time behavior. We investigate whether an analogous approximation principle holds for optimization over probability measures, where the objective is a functional defined on the Wasserstein space P2. The nonlinear geometry and infinite-dimensional nature of P2 prevent a direct extension of the classical Euclidean theory. Using Lions differentiability, we lift the problem to a linear Hilbert space, where higher-order differential calculus becomes available. We then construct a Gaussian random-field approximation whose velocity field matches the mean and covariance of the original stochastic gradient. By exploiting this moment matching through higher-order Taylor expansions, we show that the Gaussian approximation captures the SGD dynamics with second-order weak accuracy. Our result provides a rigorous foundation for replacing sample-driven randomness by analytically tractable Gaussian fluctuations in stochastic optimization over probability measures.

0.24

Trust-Region Optimization for Smooth Potential-Interaction Energies in Wasserstein Space

cs.LG stat.MLmath.OCmath.PR ·2026-10-06 ·You Wan, Ting Gao, Jinqiao Duan
theory similarity 0.54
Abstract · 24 pages, 4 figures, 1 table; code and data in the ancillary files

Finding low-energy configurations of interacting particles and approximating probability distributions lead to the minimization of potential-interaction energies in Wasserstein space. These energies can be nonconvex, making it important to exploit second-order information while controlling the reliability of local approximations. We study trust-region optimization of smooth potential-interaction energies on the Wasserstein space of probability measures with finite second moment. The method uses a quadratic model along pushforward curves, an $L^2(ρ)$ step radius, and a Steihaug-Toint subsolver with an explicit self-adjoint second-variation operator. A ratio test determines acceptance and guides the radius update. Under a lower energy bound and globally bounded Hessians of the potential and interaction kernel, we prove that the objective is nonincreasing, the Wasserstein-gradient norms converge to zero, and an $\varepsilon$-stationary iterate is reached within $O(\varepsilon^{-2})$ total outer trials, including rejected trials. If the potential is quadratically coercive, every weak accumulation point is stationary. The analysis applies to arbitrary initial measures with finite second moment. For empirical measures, the iteration is a finite-dimensional trust-region method in the $L^2(ρ_N)$ inner product, with complexity constants independent of particle number and dimension when the initial objective gaps are uniformly bounded. Numerical experiments include a smooth soft-particle energy, maximum-mean-discrepancy minimization for non-Gaussian targets, component ablations, and scaling studies in particle number and dimension.

0.27

Schrödinger Bridges over Kinetic Swarming Models

eess.SY math.OCmath-phstat.ML ·2026-08-26 ·Asmaa Eldesoukey, Md Zulfiqur Haider, Italo Napolitano, Yongxin Chen, Abhishek Halder
similarity 0.54
Abstract

Paradigmatic interaction models explain how collective behaviors can emerge in complex systems from interactions among the constituent agents. In bio-inspired swarms, however, interactions alone may not suffice to bring the population to a desired aggregate configuration within a prescribed time horizon, as needed in applications ranging from targeted therapy to collective transport and emergency evacuation. In the present work, we consider finite-horizon minimum-energy collective steering for inertial swarms that are subject to stochastic disturbances. We focus on the mean-field representations of these multi-agent systems driven by Cucker--Smale alignment or Morse attraction--repulsion interactions. Our objective is to steer the swarm between prescribed endpoint distributions using a state-feedback control, where the endpoint specifications can be full phase-space distributions (positions and velocities) or position marginals alone. Our formalism is rooted in the theory of Schrödinger bridges, which has inspired contemporary developments spanning statistical inference, biological modeling, stochastic control, and generative learning. Within the bridges framework, the uncontrolled interacting stochastic dynamics are viewed as a prior model, and the optimal control as the minimum-energy corrective drift needed to realize the prescribed distributions. We derive nonlinear, coupled necessary optimality systems with a time-symmetric structure reminiscent of classical Schrödinger bridges, and propose nested fixed-point schemes to numerically solve them. Numerical examples show that the obtained optimal control (corrective drift) can dynamically exploit or counteract the interaction forces, depending on whether the latter are favorable or adversarial to the steering task.

0.52