CS Paper Radar

Distributed Consensus, Optimization, and Learning under Communication Constraints

Zheng Chen, Nicolò Michelusi, Themistoklis Charalambous, Kaibin Huang

cs.IT cs.DCeess.SP ·submitted 2026-11-04 13:50 UTC · v1
published

Abstract

Many modern engineering applications require distributed problem-solving in which multiple nodes in a network cooperate to achieve a common optimization objective. Numerous distributed optimization and learning algorithms rely on consensus-based distributed coordination to ensure network-wide agreement via iterative local interactions. In these algorithms, the accuracy, frequency, and reliability of information exchange among network nodes directly influence system performance, including convergence behavior, training delay, and model accuracy. This tutorial article adopts a bottom-up approach to present the fundamentals of consensus averaging and distributed optimization methods, and highlights the impact of network connectivity, information flow, and communication uncertainty on network cooperation and optimization performance. Building on these theoretical insights, we discuss key design principles for communication-constrained distributed learning across diverse network topologies and review common approaches to communication-efficient learning, particularly in wireless networks. Despite substantial progress, further efforts are needed to bridge the gap between theory and practice and fully realize the potential of cooperative and networked intelligence.

Comment
20 pages, 3 figures, to appear in IEEE JSAC

Closest papers

Nearest neighbours by meaning across the whole index.

Consensus-based Decentralized Distributed Swarm Learning with Heterogeneous Big Data

cs.DC ·2026-09-10 ·Zhuoyu Yao, Dong Yang, Yue Wang, Songyang Zhang, Yingshu Li +2 more
similarity 0.70
Abstract

Artificial intelligence increasingly relies on large-scale, distributed, and heterogeneous data collected by edge devices. However, the practice of edge intelligence remains challenging due to non-convex objectives, data heterogeneity, and complex wireless network topology. To address these issues, this paper proposes a consensus-based decentralized distributed swarm learning (CD-DSL) framework for wireless edge networks. Our CD-DSL integrates consensus optimization with particle swarm optimization (PSO), by reaching the model consensus among neighboring devices while leveraging the PSO exploration and exploitation. The consensus mechanism supports decentralized coordination without raw-data exchange, while PSO-inspired updates utilize historical and neighbor-shared experience to enhance exploration for non-convex optimization, improve robustness to data heterogeneity, and accelerate convergence. We further develop an adaptive neighbor-mixing strategy that learns performance-aware consensus weights, improving decentralized collaboration among heterogeneous edge devices. Theoretical analysis establishes that CD-DSL maintains participant consistency and achieves non-ergodic convergence to a neighborhood of a stationary point under non-convex objectives. Experimental results show that CD-DSL can mitigate the performance degeneration of existing decentralized baselines caused by heterogeneous data.

0.46

KKT-Perturbation-Aware Communication for Fully Distributed Online Convex Optimization with Nonseparable Coupling

cs.LG ·2026-11-09 ·Zhaoye Pan, Tao Li, Yunjie Guo, Guanpu Chen
similarity 0.67
Abstract

Nonseparable costs and coupled constraints in distributed online convex optimization often arise when system performance and operational constraints depend on the joint configuration of all agents and cannot be decomposed into individual contributions. Such nonseparability leads to fully distributed decision-making with challenging partial local information, where each agent evaluates its local primal-dual update using estimated global information via a local belief of the joint action and the shared Lagrange multiplier to be exchanged with neighbors. While the belief consensus established by prior works ensures no-regret learning, exchanging beliefs comprising full-dimensional primal and dual states at every round imposes a communication overhead that scales with both the joint decision dimension and the network size. Existing event-triggered schemes typically activate communication according to state variation or consensus error. Under nonseparable coupling, however, errors of equal magnitude can produce markedly different perturbations after passing through the KKT operator. We therefore propose a randomized KKT-perturbation-aware communication mechanism, where the triggering of each agent's primal and dual communication channels depends on their impact on the KKT operator. The proposed mechanism transfers the adaptive-sampling principle of coordinate descent from allocating computation across decision blocks to allocating communication across information channels: channels with larger estimated perturbations receive greater priority for refresh. Under standard regularity assumptions, the resulting algorithm achieves $O({T}^{2/3})$ expected static regret and $O({T}^{2/3})$ expected cumulative constraint violation. Numerical experiments demonstrate improved learning and constraint-enforcement performance per transmitted bit over belief consensus baselines.

0.15

Distributed Optimization with Streaming Data: A Temporal Weighting Perspective

eess.SP cs.AIcs.LGeess.SY+1 ·2026-08-10 ·Muhammad Faraz Ul Abrar, Nicolò Michelusi, Erik G. Larsson
similarity 0.67
Abstract

Optimization theory is a widely used tool for intelligent decision-making. While classical optimization deals with fixed, time-invariant objective functions, many modern applications operate in dynamic environments where data arrive sequentially, and the learning objective evolves over time, often under decentralized data and communication constraints. Motivated by these trends, we study decentralized optimization from streaming data through a structured time-varying formulation in which the global objective is a temporally weighted average of losses observed across the network. We analyze multi-iteration decentralized first-order methods, including decentralized gradient descent. For strongly convex and smooth losses, we develop guarantees for the Euclidean-norm \emph{tracking error} through a contraction-mapping viewpoint. The resulting bounds decompose the tracking error into a fixed-point tracking component and a bias term induced by decentralization and data heterogeneity. We specialize our analysis to uniform and exponentially discounted weights, as well as their finite-memory \emph{windowed} counterparts. The bounds explicitly characterize the roles of the temporal weighting rule, per-step iteration budget, step size, and network connectivity. Uniform weighting yields a vanishing fixed-point tracking contribution of order $\mathcal O(1/t)$, whereas discounted and windowed strategies generally induce non-vanishing tracking floors governed by the discount factor and effective memory, respectively. In all cases, decentralization induces an additional non-zero bias floor under a constant step size. Numerical experiments illustrate the predicted trends.

0.04

Uncertainty-Aware Distributed Radio Map Estimation in Decentralized Wireless Networks

eess.SP ·2026-10-20 ·Dong Yang, Zhuoyu Yao, Yue Wang, Songyang Zhang, Yingshu Li +2 more
benchmark similarity 0.67
Abstract · 6 pages, 4 figures

Federated learning based radio map estimation (RME) approaches rely on frequent exchange of high-dimensional model updates and often require a central server for collaboration coordination and update aggregation. This incurs prohibitively high communication overhead, rendering them impractical for bandwidth-constrained mobile and wireless edge networks. Moreover, in practical mobile sensing scenarios, sparse and unevenly collected measurements lead to estimation uncertainty, which is critical for guiding RME and collaboration among distributed nodes. To address these challenges, this paper proposes an uncertainty-aware distributed radio map estimation (UA-D-RME) framework for decentralized networks. In UA-D-RME, each node locally infers a probabilistic model from its sparse measurements to estimate both its local radio map and the associated estimation uncertainty. Instead of exchanging raw data or model updates, neighboring nodes communicate only low-dimensional statistical information, including radio map and uncertainty measures over their overlapping monitoring areas. An uncertainty-aware statistical consensus mechanism is then developed to iteratively align probabilistic models in neighborhood by ensuring consistency on estimation confidence. Theoretical analysis establishes the convergence of the proposed consensus mechanism and characterizes the distributed generalization performance of the proposed UA-D-RME method. Experiments on urban radio map datasets verify that UA-D-RME achieves desired reconstruction accuracy comparable to the centralized RME benchmark while reducing communication costs and without exchanging raw measurements.

0.37

Budget-Constrained Multi-Consensus Decentralized Gradient Descent

eess.SY math.OC ·2026-10-06 ·Shuyi Ren, Nicol`o Michelusi, Erik G. Larsson
hot direction similarity 0.66
Abstract

We investigate decentralized gradient descent (DGD) with emphasis on efficient communication and computation resource utilization under budget constraints. As a first step toward the broader communication-computation allocation problem, we consider and analyze a \textit{multi-consensus decentralized gradient descent} (mcDGD) scheme, where the number of consensus rounds and the stepsize are allowed to vary across iterations. Building on a unified analytical framework for DGD, we derive finite-time convergence bounds that explicitly characterize the interaction between consensus quality and optimization dynamics. Our analysis requires only convexity of the local objective functions while assuming smoothness and strong convexity of the global objective. The resulting bounds enable a principled consensus-allocation strategy under resource constraints, for which we show that equal allocation of consensus rounds across iterations is optimal under our stepsize rule, up to integer rounding. Numerical experiments corroborate the theoretical findings and demonstrate favorable communication-computation tradeoffs compared with existing multi-consensus decentralized optimization baselines.

0.30

Test-Time Collaborative Classification over Multi-Agent Networks

cs.MA ·2026-08-25 ·Ping Hu, Mert Kayaalp, Ali H. Sayed
similarity 0.66
Abstract

The increasing heterogeneity of multi-agent systems poses significant challenges for jointly training a global model across agents. At the same time, cooperative inference between agents has long been recognized as a powerful mechanism for distributed decision making over networks. Motivated by these observations, we propose a collaboration framework for distributed binary classification over multi-agent networks, where a set of independently trained agents, potentially differing in architecture, feature space, or modality, coordinate their actions during test time to form collective predictions. This coordination is achieved by exchanging local decision statistics through a distributed learning protocol. We develop a theoretical and experimental study of this independent training and cooperative inference paradigm, and examine its performance under different communication budgets and distributed learning rules. We establish classification error guarantees under sufficient, finite-round, and finite-precision communication, together with PAC-style generalization bounds. These results capture the influence of model heterogeneity, network topology, combination policy, and communication constraints on prediction accuracy. Taken together with the experimental results, they reveal both the price of independent training and the benefit of collective prediction for the proposed distributed decision making framework with models learned from data.

0.30

Train Locally, Learn Together: Distributed Multi-Agent Training via ADMM

cs.AI ·2026-10-17 ·Yiwei Chen, Changsheng Wang, Yicheng Lang, Qing Qu, Alfred O. Hero +1 more
similarity 0.66
Abstract

Multi-agent systems (MAS) can outperform individual language-model agents by combining specialized roles, diverse reasoning processes, and collaborative decision-making. However, collaborative MAS training is typically centralized, requiring a common trainer to aggregate role-local trajectories and learning signals and maintain model and optimization states. This motivates a distributed multi-agent learning paradigm in which agents improve system-level capabilities through role-local learning and decentralized communication, paving the way toward scalable, resilient, and self-organizing MAS. To this end, we introduce, to the best of our knowledge, the first optimization-grounded framework for distributed MAS training based on consensus alternating direction method of multipliers (ADMM). We formulate collaborative MAS learning as a consensus-constrained finite-sum optimization problem, naturally lending itself to distributed optimization via ADMM. The proposed approach alternates role-local learning with inter-agent consensus using distributed communication protocols such as ring all-reduce, enabling joint optimization while keeping each agent's data and training state local. We evaluate our framework on distributed safety alignment and general reasoning tasks. Under compromised-agent settings, ADMM can reduce HarmBench attack success rate (ASR) by up to 26.4%, while achieving reasoning gains comparable to centralized training. These results show that distributed MAS training retains the benefits of centralized joint optimization without requiring a centralized trainer.

0.19

Non-Coherent Over-the-Air Distributed Consensus: Algorithm Design and Power Control Optimization

cs.IT eess.SPeess.SY ·2026-11-02 ·Yuhang Deng, Zheng Chen, Erik G. Larsson
publishedtheory similarity 0.65
Abstract · 13 pages, 4 figures. Extended version of a paper published in Proc. IEEE ICC 2025

Non-coherent over-the-air (NCOTA) aggregation offers a scalable and communication-efficient approach to distributed average consensus in wireless networks, without requiring instantaneous channel state information (CSI) or phase alignment. However, persistent aggregation errors pose fundamental challenges in achieving exact average consensus under nominal distributed averaging protocols. In this paper, we develop an NCOTA consensus framework that accommodates row-stochastic mixing, without requiring the conventional average-preservation property of doubly stochastic mixing matrices. Building on a distributed optimization formulation of average consensus, we propose a decentralized projected gradient descent with row-stochastic mixing (DPGD-RS) algorithm and establish its mean-square convergence to the exact average of the agents' initial states. We further derive a closed-form upper bound on the mean-square consensus error, characterizing the effects of aggregation uncertainty, network connectivity, and mixing imbalance. Guided by this analysis, we propose a spectral-reshaping power control (SR-PC) scheme that jointly optimizes transmit powers and receive-scaling factors to accelerate convergence by balancing aggregation accuracy and network mixing. Numerical results validate the theoretical convergence guarantees and demonstrate that the proposed power control scheme significantly accelerates convergence compared with full-power transmission and spectral-radius-minimization benchmarks.

0.40