CS Paper Radar

Global Sinkhorn Convergence and Spectral Stability of Matrix Scaling

Kun He

cs.DS ·submitted 2026-11-06 04:17 UTC · v1
theory

Abstract

We give a global spectral bound for the Sinkhorn--Knopp (SK) algorithm. Let $A$ be a row-stochastic nonnegative $n\times n$ matrix admitting a finite doubly stochastic scaling $B$. Define \[ g=1-s_2(B)^2>0, \qquad D=\min\left\{\|(u,v)\|_\infty: B=\operatorname{diag}(e^u)A\operatorname{diag}(e^{-v})\right\}. \] From the standard initialization, SK achieves total marginal $\ell_1$ error $\varepsilon$ in \[ O\!\left(\frac{D+\log n+\log(1/\varepsilon)}{g}\right) \] iterations. A scale-invariant potential contracts globally according to the spectrum of $B$. The linear dependence on $D$ is necessary in the fixed-gap regime: we construct strictly positive inputs with constant input and endpoint spectral gaps, $D=Θ(n)$, and an $Ω(n)$ iteration lower bound at error $n^{-2}$. Our upper bound gives $O(n)$ iterations on this family. We next give spectral certificates for scalability and short scaling distance. For a row-stochastic input $A$ with positive column sums $c_j$, set \[ K=\frac{\max_jc_j}{\min_jc_j},\qquad σ=s_2\!\left(A\operatorname{diag}(c)^{-1/2}\right). \] The input-only condition $\sqrt K\,σ<1$ certifies exact scalability and polynomial scaling distortion, with $D=O(\log n)$ for fixed $K$ and a fixed positive spectral margin. Given a finite scaling $B$, the weaker condition $\sqrt K\,σs_2(B)<1$ gives the same structural bounds. With a constant endpoint gap, these bounds imply logarithmic SK iteration complexity. Boundary constructions show that the strict spectral inequalities cannot uniformly be replaced by nonstrict ones.

Comment
71 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Tight Nonasymptotic Local Convergence of Sinkhorn-Knopp

cs.LG math.OCstat.ML ·2026-08-12 ·Wenzhi Gao, Zhaonan Qu, Yinyu Ye, Madeleine Udell
similarity 0.66
Abstract

We revisit the Sinkhorn-Knopp (SK) algorithm for the matrix scaling problem. Despite extensive literature on the global convergence of SK and its variants, its local linear convergence behavior remains less understood. We address this gap by providing the first nonasymptotic local analysis of SK that matches the rate obtained from existing asymptotic Jacobian-based arguments. We show that under certain connectivity conditions, SK is a polynomial-time algorithm for doubly stochastic matrix scaling. With the developed tools, we showcase the local suboptimality of SK and provide accelerated variants. Finally, for dense matrices, we improve the complexity of existing first-order matrix scaling algorithms from $O(\tfrac{n^{7/3}}{\varepsilon^{2/3}})$ to $O(\tfrac{n^{9/4}}{\sqrt{\varepsilon}})$.

0.34

Uniform Statistical Convergence of Empirical Sinkhorn Potentials with Exponential and Polynomial Dependence on the Regularization Parameter

cs.LG stat.MLmath.OC ·2026-08-29 ·Denis Belomestny
theory similarity 0.61
Abstract

We study the empirical Sinkhorn estimator of the entropic optimal transport potentials under the uniform loss. Since the potentials are only unique up to additive constants, we measure the error using the quotient supremum norm, defined as $d_\infty([u],[v]) = \inf_{a\in\mathbb{R}}\|u-v-a\|_\infty$. For a fixed regularization parameter $\varepsilon>0$, we establish a non-asymptotic statistical rate of $n^{-1/2}$. This is achieved by combining the Birkhoff-Hopf contraction theorem with entropy bounds on normalized kernel sections. However, the constant in this bound grows exponentially with $1/ε$. To improve this, we isolate geometric conditions under which the empirical estimator maintains the $n^{-1/2}$ rate but features polynomial dependence on $1/\varepsilon$. The key requirement is a polynomial residual-stability estimate for the population Sinkhorn map. We provide sufficient criteria for this, including a polynomial contraction property and a local inverse estimate. Furthermore, we introduce two rigorously verifiable model classes an $\varepsilon$-weak residual-interaction class obtained after separable centering and another based on connected tight-edge graphs for fixed discrete costs where the polynomial rate is guaranteed without relying on abstract resolvent assumptions. Finally, we establish matching minimax lower bounds demonstrating that the $\varepsilon n^{-1/2}$ rate cannot be uniformly improved in the bounded-interaction regime.

0.34

Rank-1-perturbed trickledown theorems: Mixing time of Glauber dynamics for the Sherrington-Kirkpatrick model up to $β\leq \frac{1}{2}+\varepsilon$

cs.DS math-phmath.PR ·2026-09-11 ·Mathews Boban, Anqi Li, Shayan Oveis Gharan
similarity 0.61
Abstract · 34 pages

We introduce a new family of trickledown theorems, a.k.a., local to global technique to bound the spectral gap of the Glauber dynamics for multi-state spin systems. In this technique instead of upper-bounding the influence matrix of a link of co-dimension 2 by $λI$ (where $λ$ is the second eigenvalue of the link), we upper-bound the influence matrix after a carefully chosen rank-1 shift. The rank-1 shift allows for a significantly smaller upper-bound but it comes at the cost of bounding the average loss due to rank-1 perturbations. As an application we use this method to show that the natural Glauber dynamics mixes in polynomial time to generate samples from the Sherrington-Kirkpatrick model for $β\leq \tfrac{1}{2}+\varepsilon$, for an absolute constant $\varepsilon>0$. At the heart of the proof we manage to bound the loss due to rank-1 perturbations by averaging over all links of co-dimension 2.

0.39

Free-Probabilistic State Evolution and Random Matrix Discrepancy

cs.DS math.PR ·2026-09-10 ·August Y. Chen, Ahmed El Alaoui
similarity 0.59
Abstract · Revised version contains stronger results on random matrix discrepancy, further minor edits, and an updated bibliography

Let $A_1,\ldots,A_n$ be independent $d \times d$ real symmetric Gaussian random matrices, and consider the linear operator $A(x) = n^{-1/2}\sum_{i=1}^n x_iA_i$, $x\in \mathbb R^n$. We construct an iterative algorithm in the Approximate Message Passing family which iterates over $A$ and its adjoint $A^*$, and establish a state evolution result which characterizes its behavior in the limit $d\to \infty, 2n/d^2 \to α$ in terms of a correlated Gaussian-semicircular process in a free probability space, in the sense of strong convergence of operators. We then apply this iteration to the random matrix discrepancy problem which asks for a binary vector $x \in \{-1,+1\}^n$ such that $A(x)$ has a small operator norm. Our algorithm achieves an operator norm $2σ(α)$, for a standard deviation $σ(α)<1$ for all $α>0$, and $σ(α) = O(1/\sqrt{α\log α})$ for large $α$. This resolves the algorithmic question of Kunisky-Zhang (2023) and Maillard (2025).

0.25

A Self-Generated Floor on Adam's $\varepsilon$: Evidence from Power-Law Matrix Sensing

cs.LG ·2026-11-01 ·Lngel Kyo
similarity 0.59
Abstract · 23 pages, 3 figures, 5 tables

In nonconvex Burer--Monteiro low-rank matrix sensing of $U^\ast \in \mathbb{R}^{V \times D}$ under power-law covariance $Σ= \mathrm{diag}(j^{-s})_{j=1}^V$, the precision-wall approximation predicts that Adam stalls when the smallest non-gauge Hessian eigenvalue drops below its denominator floor, giving $s^\ast(V,D) = [\log(1/\varepsilon_{\mathrm{eff}}) - \log V]/\log(VD) + δ_{\mathrm{Adam}}$. If $\varepsilon_{\mathrm{eff}}$ were set by $\varepsilon_0$, lowering $\varepsilon_0$ from $10^{-10}$ to $10^{-12}$ would raise $s^\ast$ by $0.95\text{--}1.33$. Instead, across six values of $\varepsilon_0$ two decades apart, $10^{-2}$ to $10^{-12}$, the boundary follows the $\varepsilon_0$ slope down to $10^{-6}$, bends near $10^{-8}$, and is flat below $10^{-8}$, while warm-started L-BFGS and Gauss--Newton reach machine precision on stalled instances. We trace this saturation to a self-generated dynamical floor $\varepsilon_{\mathrm{dyn}} \propto α$: high-curvature modes oscillate persistently near the adaptive edge of stability with amplitude $\propto α(1-β_1)/(1+β_1)$ and leak gradient fluctuations of scale $κα$ into rare modes via measurement coupling, pinning $\sqrt{\hat{v}_V} \approx κα\gg \varepsilon_0$. Fitting a single threshold-crossing wall parameter $\varepsilon_{\mathrm{dyn}} = 4.76\times 10^{-9}$ (with the detection lag frozen at its companion calibration) predicts the saturated failure boundary for $D \ge 4$ across five $(V,D)$ scales to RMS $0.098$ (breaking down at $D = 2$), while shrinking $\varepsilon_0$ below $10^{-8}$ is ineffective and lowering the step size $α$ reduces the floor directly.

0.26

Stable and Online Algorithms for Random Matrix Discrepancy

cs.DS cs.CCcs.DMmath.CO+1 ·2026-10-01 ·Eren C. Kızıldağ, Shuangping Li
theoryhighly cited similarity 0.58
Abstract

We study the average-case matrix discrepancy problem: given independent normalized $d\times d$ Gaussian orthogonal ensemble matrices $A_1,\dots,A_N$ and a fixed margin $κ>0$, find signs $σ_1,\dots,σ_N\in\{-1,1\}$ such that the operator norm of $\sum_{i=1}^N σ_i A_i$ is at most $κ\sqrt{N}$. Focusing on the proportional regime $N/d^2\to τ\in(0,\infty)$ as $d\to\infty$ followed by the small-margin limit $κ\downarrow 0$, we characterize the density required by stable offline algorithms and by online algorithms. In the offline setting, we construct a polynomial-time \emph{recenter-and-round} algorithm that is noise-stable and succeeds whenever $τ=Ω(\frac{1}{κ^2\log(1/κ)})$, along with a matching lower bound for all stable algorithms. In the online setting where each sign must be chosen irrevocably upon observing the corresponding matrix, we determine the exact limiting performance of the \emph{Frobenius-greedy} algorithm, establishing that it succeeds when $τ>τ_{\rm FG}(κ)\sim \fracπ{4κ^2}$, as well as a matching lower bound for all online algorithms by conditioning on a revealed prefix. At the core of our algorithms lies rotational symmetry, which enables us to transfer Frobenius norm control into operator norm guarantees. Together, our results identify the algorithmic phase transition points for random matrix discrepancy: $Θ(\frac{1}{κ^2\log(1/κ)})$ for stable offline algorithms and $Θ(\frac{1}{κ^2})$ for online algorithms. Both thresholds lie far above the satisfiability scale $Θ(\log(1/κ))$, as shown by Maillard~\cite{maillard2025}.

0.31

On the Spectral Dependence of Decentralized Stochastic Gradient Tracking

eess.SY math.OC ·2026-11-05 ·Sulaiman A. Alghunaim, Haoyuan Cai, Ali H. Sayed
theory similarity 0.57
Abstract

We study how the spectral geometry of the mixing matrix enters the convergence of decentralized stochastic gradient tracking (GT) for smooth strongly convex problems. General bounds contain the factor $Δ_λ^{-3}$ in the higher-order stochastic term, where $Δ_λ$ is the absolute spectral gap. We show that, even for two agents with identical quadratic objectives, the exact steady-state mean-square disagreement scales as $α^2σ^2Δ_λ^{-3}$ as the nontrivial eigenvalue approaches $-1$, provided the stepsize $α$ is sufficiently small relative to the gap. Here $σ^2$ is the gradient-noise variance. By unrolling the disagreement recursion, we establish a stochastic bound that averages all nontrivial eigenvalue contributions for diagonalizable mixing matrices, up to an eigenvector conditioning factor. As a real or complex eigenvalue approaches the unit circle away from $1$, its contribution grows as the inverse cube of its distance to the circle; growth can be slower near $1$. For symmetric mixing with gaps $p$ at $1$ and $ν$ at $-1$, the weighted sum $θ_1/p+θ_2/ν^3$ replaces the prior factor $(Δ_λν^2)^{-1}$. For $N$ agents, $Nθ_1$ and $Nθ_2$ count the nonnegative and negative nontrivial eigenvalues, respectively. On a symmetric family, a finite-horizon lower bound for the average iterate matches the worst-case factor $p^{-1}+ν^{-3}$ up to universal constants. Numerical experiments confirm the predicted scaling.

0.22

Beyond the Bochner Method: Optimal Spectral Gaps for Block Dynamics of Spin Systems like the $O(N)$ Model

cs.DS math-phmath.COmath.PR ·2026-11-03 ·Jonathan Leake, Anqi Li
theory similarity 0.57
Abstract · 42 pages

Let $μ$ be a probability distribution on a spin system with $n$ sites and possibly infinite spin spaces. We prove local-to-global spectral-gap bounds for block Glauber dynamics. Given a symmetric, entrywise nonnegative matrix $A$ with zero diagonal and $λ_{\max}(A)<1$ encoding local spectral bounds for a spin system, we derive global spectral-gap bounds for block dynamics on its marginal distributions. These bounds are sharp: the Gaussian system with covariance $(I-A)^{-1}$ attains equality for every choice of disjoint blocks. Our proof extends the Bochner method through an abstract inequality for orthogonal projections on Hilbert spaces. As an application, we prove that Glauber dynamics for the $O(N)$ model, with $N\geq 2$, arbitrary external fields, and couplings of either sign, has spectral gap at least $(1-\|J_{\mathrm{abs}}\|_{\mathrm{op}}/N)/n$, where $J$ is the interaction matrix and $J_{\mathrm{abs}}=(|J_{ij}|)$. We also obtain spectral bounds for swap walks and local spectral expansion of partite simplicial complexes. For complexes formed from disjoint groups of ranks in random flags of subspaces over $\mathbb F_q$, we obtain a one-sided local spectral expansion bound of $2\sqrt q/(q+1)$. This bound is independent of the number and sizes of each group, and is nontrivial for every prime power $q$. Finally, for random forests $F$ of any finite graph with arbitrary positive edge weights, we prove that \[ \Pr(e,f\in F)\le \frac{4}{3} \Pr(e\in F)\Pr(f\in F) \] for distinct edges $e,f$, improving the factor 2 supplied by general log-concavity bounds for matroid independent sets. The argument combines our extended Bochner method with the analysis of an auxiliary real stable polynomial.

0.28