CS Paper Radar

Exact Average Consensus under Noisy Communication Links: A Decentralized Gradient Perspective

Yuhang Deng, Zheng Chen, Erik G. Larsson

eess.SY cs.ITmath.OC ·submitted 2026-09-23 13:22 UTC · v1
publishedtheory

Abstract

We study the distributed average consensus problem under persistent link-level disturbances modeled as a martingale difference sequence with uniformly bounded conditional second moments. Under such disturbances, the standard stochastic-approximation-based linear iteration with diminishing stepsizes drives the network to consensus on an unbiased random variable with non-vanishing variance instead of the exact initial average. To understand and resolve this limitation, we develop an anchoring-based mechanism derived from a decentralized gradient descent formulation and study the effect of incorporating a decaying anchoring term that continuously pulls each agent state toward its initial value. This perspective provides an intuitive interpretation of how state anchoring counteracts disturbance accumulation. Under standard summability conditions, we prove that the resulting algorithm achieves exact average consensus almost surely. Furthermore, this decentralized gradient perspective offers a unifying framework for several related methods and an interpretable design principle for exact average consensus under persistent disturbances.

Comment
8 pages, 2 figures. Accepted to the 2026 IEEE 65th Conference on Decision and Control (CDC 2026)

Closest papers

Nearest neighbours by meaning across the whole index.

Stochastic Gradient Tracking over Time-Varying Networks: One-Step Lyapunov Analysis

cs.DC math.OCeess.SY ·2026-08-17 ·Sulaiman A. Alghunaim
similarity 0.68
Abstract

We study decentralized stochastic gradient tracking over a time-varying network of $N$ agents under a uniform window-mixing condition. Products of $τ$ consecutive doubly stochastic mixing matrices contract disagreement by a factor $λ<1$, although individual matrices need not contract disagreement strictly and individual communication graphs may be disconnected. We construct a time-varying quadratic norm that turns this window contraction into an exact one-step Lyapunov identity. This leads to coupled one-step recursions for the centroid and disagreement errors, without unrolling the dynamics over communication windows. For smooth strongly convex objectives, the leading stochastic term is $\widetilde{\mathcal O}(1/(NK))$; for smooth convex objectives, it is $\mathcal O(1/\sqrt{NK})$. Both match their centralized mini-batch counterparts and yield linear speedup after a network-dependent transient.

0.39

Non-Coherent Over-the-Air Distributed Consensus: Algorithm Design and Power Control Optimization

cs.IT eess.SPeess.SY ·2026-11-02 ·Yuhang Deng, Zheng Chen, Erik G. Larsson
publishedtheory similarity 0.65
Abstract · 13 pages, 4 figures. Extended version of a paper published in Proc. IEEE ICC 2025

Non-coherent over-the-air (NCOTA) aggregation offers a scalable and communication-efficient approach to distributed average consensus in wireless networks, without requiring instantaneous channel state information (CSI) or phase alignment. However, persistent aggregation errors pose fundamental challenges in achieving exact average consensus under nominal distributed averaging protocols. In this paper, we develop an NCOTA consensus framework that accommodates row-stochastic mixing, without requiring the conventional average-preservation property of doubly stochastic mixing matrices. Building on a distributed optimization formulation of average consensus, we propose a decentralized projected gradient descent with row-stochastic mixing (DPGD-RS) algorithm and establish its mean-square convergence to the exact average of the agents' initial states. We further derive a closed-form upper bound on the mean-square consensus error, characterizing the effects of aggregation uncertainty, network connectivity, and mixing imbalance. Guided by this analysis, we propose a spectral-reshaping power control (SR-PC) scheme that jointly optimizes transmit powers and receive-scaling factors to accelerate convergence by balancing aggregation accuracy and network mixing. Numerical results validate the theoretical convergence guarantees and demonstrate that the proposed power control scheme significantly accelerates convergence compared with full-power transmission and spectral-radius-minimization benchmarks.

0.40

Distributed Stochastic Approximation Algorithms and Heavy-Tailed Age of Information

cs.DC math.OCcs.MAcs.NI+1 ·2026-09-23 ·Adrian Redder, Arunselvan Ramaswamy, Holger Karl
similarity 0.64
Abstract

Algorithms in multi-agent systems such as federated learning, mobile robotic swarming, and consensus control can be designed and analyzed as distributed stochastic approximation algorithms. Such algorithms involve information exchanges between agents for various computations. The freshness of the information can be quantified using the Age of Information (AoI) metric. Consider robotic teams operating in highly obstructed geographical settings, such as subterranean or dense urban environments. Because of spatial disconnections, AoI has empirically been observed to be heavy-tailed with unbounded moments. However, most analyses assume AoI with bounded moments, creating a gap between theory and practice. To the best of our knowledge, ours is the first analysis under general heavy-tailed AoI with potentially infinite mean. We study the stability (almost sure boundedness of the distributed iterates) and convergence of multi-agent systems that are strictly dissipative in the scaling limit (system at ``infinity''). Examples include most gradient-based and consensus algorithms under the Robbins-Monro step-size regime.

0.20

Budget-Constrained Multi-Consensus Decentralized Gradient Descent

eess.SY math.OC ·2026-10-06 ·Shuyi Ren, Nicol`o Michelusi, Erik G. Larsson
hot direction similarity 0.64
Abstract

We investigate decentralized gradient descent (DGD) with emphasis on efficient communication and computation resource utilization under budget constraints. As a first step toward the broader communication-computation allocation problem, we consider and analyze a \textit{multi-consensus decentralized gradient descent} (mcDGD) scheme, where the number of consensus rounds and the stepsize are allowed to vary across iterations. Building on a unified analytical framework for DGD, we derive finite-time convergence bounds that explicitly characterize the interaction between consensus quality and optimization dynamics. Our analysis requires only convexity of the local objective functions while assuming smoothness and strong convexity of the global objective. The resulting bounds enable a principled consensus-allocation strategy under resource constraints, for which we show that equal allocation of consensus rounds across iterations is optimal under our stepsize rule, up to integer rounding. Numerical experiments corroborate the theoretical findings and demonstrate favorable communication-computation tradeoffs compared with existing multi-consensus decentralized optimization baselines.

0.30

Distributed Zeroth-Order Composite Finite-Sum Optimization via Incremental Variance Reduction

eess.SY math.OC ·2026-10-30 ·Silan Zhang, Yujie Tang
similarity 0.63
Abstract

We study decentralized composite finite-sum optimization over a fixed connected undirected network, where agents access local component functions through function evaluations and share a possibly nonsmooth convex regularizer. In this setting, standard two-point gradient estimators can retain nonvanishing variance at a solution, complicating their integration with gradient tracking. We propose Distributed Zeroth-Order Incremental Variance Reduction (Distributed ZIVR), which combines incrementally updated component-gradient estimates with a proximal gradient-tracking scheme. The method admits an implementation using a constant number of function evaluations per agent at every iteration, without intermittent full-component or full-coordinate refreshes. For convex smooth component functions and strongly convex local averages, we establish linear convergence up to a controllable finite-difference error. With the finite-difference radius chosen for the target accuracy, Chebyshev-accelerated mixing yields a per-agent oracle complexity of $\widetilde{O}(d(m+κ)\log(1/ε))$ for expected squared solution error $ε$, where $d$ is the dimension, $m$ is the number of components per agent, and $κ$ is the condition number. This matches the centralized ZIVR oracle bound, while the communication bound has only an inverse square-root dependence on the network spectral gap. Experiments on logistic and robust regression demonstrate rapid reduction of solution and consensus errors.

0.13

On the Spectral Dependence of Decentralized Stochastic Gradient Tracking

eess.SY math.OC ·2026-11-05 ·Sulaiman A. Alghunaim, Haoyuan Cai, Ali H. Sayed
theory similarity 0.63
Abstract

We study how the spectral geometry of the mixing matrix enters the convergence of decentralized stochastic gradient tracking (GT) for smooth strongly convex problems. General bounds contain the factor $Δ_λ^{-3}$ in the higher-order stochastic term, where $Δ_λ$ is the absolute spectral gap. We show that, even for two agents with identical quadratic objectives, the exact steady-state mean-square disagreement scales as $α^2σ^2Δ_λ^{-3}$ as the nontrivial eigenvalue approaches $-1$, provided the stepsize $α$ is sufficiently small relative to the gap. Here $σ^2$ is the gradient-noise variance. By unrolling the disagreement recursion, we establish a stochastic bound that averages all nontrivial eigenvalue contributions for diagonalizable mixing matrices, up to an eigenvector conditioning factor. As a real or complex eigenvalue approaches the unit circle away from $1$, its contribution grows as the inverse cube of its distance to the circle; growth can be slower near $1$. For symmetric mixing with gaps $p$ at $1$ and $ν$ at $-1$, the weighted sum $θ_1/p+θ_2/ν^3$ replaces the prior factor $(Δ_λν^2)^{-1}$. For $N$ agents, $Nθ_1$ and $Nθ_2$ count the nonnegative and negative nontrivial eigenvalues, respectively. On a symmetric family, a finite-horizon lower bound for the average iterate matches the worst-case factor $p^{-1}+ν^{-3}$ up to universal constants. Numerical experiments confirm the predicted scaling.

0.22

Decentralized SGD under Heavy-Tailed Noise: Optimal Convergence Rates and the Role of Gradient Clipping

cs.LG math.OCcs.MA ·2026-10-07 ·Aleksandar Armacki, Haoyuan Cai, Ali H. Sayed
similarity 0.61
Abstract · 36 pages, 5 figures, 2 tables

Heavy-tailed noise has been widely observed in modern machine learning, motivating the use of methods like gradient clipping and normalization. While these methods are well understood in centralized settings, much less is known in decentralized ones, where applying a nonlinearity to local gradients affects both optimization and consensus. Recent works on decentralized non-convex optimization have studied both clipping and normalization under heavy-tailed noise, with clipping yielding suboptimal rates and normalization needing local momentum or mini-batches to converge. This raises the question: can a baseline decentralized method using a nonlinearity achieve optimal convergence rates under heavy-tailed noise? We answer affirmatively with clipped decentralized SGD ($\mathtt{DSGD}$). For smooth non-convex costs under bounded $p$-th moment noise, $p \in (1,2]$, we show that clipped $\mathtt{DSGD}$ achieves order-optimal rates both with high probability and in expectation. Moreover, we establish a linear speed-up in the number of agents, which, to our knowledge, has not been shown for decentralized methods with clipping. The key technical ingredient is a sharp analysis of the consensus gap that exploits the structure of clipping, relegating network effects to higher-order terms. Our results highlight an important distinction between clipping and normalization in decentralized settings: while normalized $\mathtt{DSGD}$ can fail to converge, clipping retains magnitude information, enabling $\mathtt{DSGD}$ to be convergent and order-optimal. Numerical experiments validate our theory.

0.42

Toward a decentralized design framework for discrete-time distributed algorithms via their emergent dynamics

eess.SY ·2026-10-21 ·Jinwook Heo, Hyungbo Shim, Jin Gyu Lee
similarity 0.61
Abstract

This paper develops a procedure for designing discrete-time distributed algorithms to achieve a desired collective behavior. Each agent combines its node and coupling updates at every iteration, in either explicit or implicit form. For this structure, we derive necessary and sufficient conditions on the local parameters for realizing a desired continuous-time blended dynamics. These conditions yield a decentralized procedure for computing the parameters. A Tikhonov-type approximation result relates the resulting agent trajectories to the blended dynamics on a rescaled time axis. Under a stability condition imposed only on the blended dynamics, we further establish uniform-in-time approximation bounds. We also characterize the trade-off between approximation accuracy and the iteration count guaranteed by these bounds. The framework is illustrated through four applications in distributed computation and estimation.

0.20