CS Paper Radar

Consensus-based Decentralized Distributed Swarm Learning with Heterogeneous Big Data

Zhuoyu Yao, Dong Yang, Yue Wang, Songyang Zhang, Yingshu Li, Zhi Tian, Zhipeng Cai

cs.DC ·submitted 2026-09-10 19:23 UTC · v1

Abstract

Artificial intelligence increasingly relies on large-scale, distributed, and heterogeneous data collected by edge devices. However, the practice of edge intelligence remains challenging due to non-convex objectives, data heterogeneity, and complex wireless network topology. To address these issues, this paper proposes a consensus-based decentralized distributed swarm learning (CD-DSL) framework for wireless edge networks. Our CD-DSL integrates consensus optimization with particle swarm optimization (PSO), by reaching the model consensus among neighboring devices while leveraging the PSO exploration and exploitation. The consensus mechanism supports decentralized coordination without raw-data exchange, while PSO-inspired updates utilize historical and neighbor-shared experience to enhance exploration for non-convex optimization, improve robustness to data heterogeneity, and accelerate convergence. We further develop an adaptive neighbor-mixing strategy that learns performance-aware consensus weights, improving decentralized collaboration among heterogeneous edge devices. Theoretical analysis establishes that CD-DSL maintains participant consistency and achieves non-ergodic convergence to a neighborhood of a stationary point under non-convex objectives. Experimental results show that CD-DSL can mitigate the performance degeneration of existing decentralized baselines caused by heterogeneous data.

Closest papers

Nearest neighbours by meaning across the whole index.

Distributed Consensus, Optimization, and Learning under Communication Constraints

cs.IT cs.DCeess.SP ·2026-11-04 ·Zheng Chen, Nicolò Michelusi, Themistoklis Charalambous, Kaibin Huang
published similarity 0.70
Abstract · 20 pages, 3 figures, to appear in IEEE JSAC

Many modern engineering applications require distributed problem-solving in which multiple nodes in a network cooperate to achieve a common optimization objective. Numerous distributed optimization and learning algorithms rely on consensus-based distributed coordination to ensure network-wide agreement via iterative local interactions. In these algorithms, the accuracy, frequency, and reliability of information exchange among network nodes directly influence system performance, including convergence behavior, training delay, and model accuracy. This tutorial article adopts a bottom-up approach to present the fundamentals of consensus averaging and distributed optimization methods, and highlights the impact of network connectivity, information flow, and communication uncertainty on network cooperation and optimization performance. Building on these theoretical insights, we discuss key design principles for communication-constrained distributed learning across diverse network topologies and review common approaches to communication-efficient learning, particularly in wireless networks. Despite substantial progress, further efforts are needed to bridge the gap between theory and practice and fully realize the potential of cooperative and networked intelligence.

0.32

Uncertainty-Aware Distributed Radio Map Estimation in Decentralized Wireless Networks

eess.SP ·2026-10-20 ·Dong Yang, Zhuoyu Yao, Yue Wang, Songyang Zhang, Yingshu Li +2 more
benchmark similarity 0.65
Abstract · 6 pages, 4 figures

Federated learning based radio map estimation (RME) approaches rely on frequent exchange of high-dimensional model updates and often require a central server for collaboration coordination and update aggregation. This incurs prohibitively high communication overhead, rendering them impractical for bandwidth-constrained mobile and wireless edge networks. Moreover, in practical mobile sensing scenarios, sparse and unevenly collected measurements lead to estimation uncertainty, which is critical for guiding RME and collaboration among distributed nodes. To address these challenges, this paper proposes an uncertainty-aware distributed radio map estimation (UA-D-RME) framework for decentralized networks. In UA-D-RME, each node locally infers a probabilistic model from its sparse measurements to estimate both its local radio map and the associated estimation uncertainty. Instead of exchanging raw data or model updates, neighboring nodes communicate only low-dimensional statistical information, including radio map and uncertainty measures over their overlapping monitoring areas. An uncertainty-aware statistical consensus mechanism is then developed to iteratively align probabilistic models in neighborhood by ensuring consistency on estimation confidence. Theoretical analysis establishes the convergence of the proposed consensus mechanism and characterizes the distributed generalization performance of the proposed UA-D-RME method. Experiments on urban radio map datasets verify that UA-D-RME achieves desired reconstruction accuracy comparable to the centralized RME benchmark while reducing communication costs and without exchanging raw measurements.

0.37

Decentralized Federated Learning for Heterogeneous Multi-Task Semantic Communication

cs.AI cs.LG ·2026-08-15 ·Lin Yin, Tiejun Lv, Weicai Li, Xi Yu, Xiaoyu He
top venue similarity 0.62
Abstract · 17 pages, 9 figures, Accepted by IEEE Transactions on Communications

Collaborative training in distributed semantic communication (DSC) networks typically relies on decentralized federated learning (DFL). However, pushing topology-agnostic aggregation into heterogeneous, multi-task environments creates a fundamental bottleneck: it drives negative transfer and overconsensus bias (OCB). This paper introduces a personalized DSC framework that cuts off this cross-task interference. At the node level, a policy-driven multi-path routing mechanism separates task-specific features from shared representations to preserve local fidelity. Across the network, we deploy a "communicationwhile- aggregation" protocol. It calibrates a column-stochastic consensus matrix using task affinities. This limits the system to absorbing complementary knowledge while actively blocking mismatched parameter updates. To bound the convergence, we derive a unified Lyapunov drift analysis. We reveal a strict Ushaped trade-off: deeper topological mixing reduces variance but amplifies structural OCB. Resolving this tension yields a closed-form expression for the optimal aggregation depth. We evaluate the proposed framework on NYU-v2, where the results reveal a clear trade-off between insufficient aggregation and excessive topological mixing. At the analytically derived optimal aggregation depth, our method achieves a 4.77% global relative improvement over the no-aggregation baseline and outperforms decentralized FedAvg, FedAMP, and heuristic max aggregation. We further evaluate the framework on Taskonomy and imperfect wireless links to examine the effects of network-size variation and wireless-link reliability.

0.35

Resilient Decentralized Wireless Federated Learning via Gradient Tracking with AdamW

cs.LG cs.DCcs.NI ·2026-08-26 ·Nguyen Van Thieu, Ti Ti Nguyen, Ons Aouedi, Vu Nguyen Ha, Symeon Chatzinotas
published similarity 0.61
Abstract · Accepted at the 2026 IEEE Global Communications Conference (GLOBECOM 2026), IoT and Sensor Networks Symposium, 7 pages, 5 figures

Wireless Internet-of-Things (IoT) edge networks require decentralized learning (DecL) methods that can operate reliably under both heterogeneous local data and communication-constrained wireless links. However, existing decentralized optimization schemes often incur substantial communication overhead and degraded performance when transmissions are constrained by strict airtime budgets, fading channels, and packet losses. This paper proposes QEF-GT-AdamW, a communication-efficient and outage-resilient algorithm for DecL over wireless communication (WCom) networks. The proposed method combines gradient tracking to mitigate the effect of non-IID data, AdamW-based adaptive optimization to improve training stability, and dual-stream biased quantization with error feedback to reduce communication payloads for both model and tracking exchanges. To address unreliable broadcast communication, the proposed framework further employs a local fallback strategy when scheduled packets are not successfully received. We explicitly model the effect of bandwidth, transmit power, airtime constraints, and fading channels on DecL performance, and establish convergence guarantees for the proposed algorithm under compressed and unreliable wireless communication. Experimental results on heterogeneous MNIST and CIFAR-10 settings show that QEF-GT-AdamW consistently improves robustness and convergence performance over representative DecL baselines while achieving favorable accuracy-communication trade-offs under limited wireless resources.

0.26

DPA: Decentralized Primal Averaging with Quasi-Global Momentum for Highly Heterogeneous Data

cs.LG cs.AI ·2026-11-12 ·Diyako Ghaderyan, Stefan Werner
top venuebenchmarkcode/data similarity 0.61
Abstract · Accepted at NeurIPS 2026. 7 figures. Code available at https://github.com/Diyako754/DPA

Decentralized training of deep learning models keeps training data local and supports on-device learning over networks. In realistic scenarios, heterogeneity across agents' local data poses an optimization challenge and can severely degrade test accuracy, especially on sparse communication graphs. Building on recent advances in centralized primal averaging, we propose Decentralized Primal Averaging (DPA), a decentralized learning algorithm that combines primal averaging with a local quasi-global momentum buffer. DPA maintains three coupled sequences for optimization, gradient evaluation, and output averaging, while using consecutive mixed iterates to construct a network-aware momentum direction without transmitting an auxiliary tracking variable. For smooth nonconvex objectives, we establish a convergence bound for DPA and identify conditions for linear speedup in the number of agents for sufficiently many iterations. We also introduce DPA-1G, a variant that uses one communication round per iteration and matches the per-iteration communication cost of DSGD-style baselines. Experiments following the GUT decentralized image-classification benchmark show that DPA compares favorably with DSGD, gradient tracking, QG-DSGDm, momentum tracking, and QG-GUTm across datasets, architectures, topologies, and heterogeneity levels, with the largest gains under high heterogeneity on sparse rings. At the same total communication cost, both DPA variants achieve higher mean test accuracy than the best baseline on CIFAR-10 and CIFAR-100 with ResNet-20. Our code is publicly available.

0.37

Adaptive Client Clustering and Coordination for Federated Learning Workflow Management in Edge Networks

cs.DC ·2026-09-27 ·Jieping Luo, Qiyue Li, Yuxuan Chen, Hang Qi, Jiaying Yin +2 more
similarity 0.61
Abstract · 15 pages

Federated learning (FL) is increasingly deployed as a managed learning service rather than as a set of isolated training jobs. In networked edge environments, dependent FL service flows must coordinate heterogeneous clients, non-IID data, fluctuating communication latency, and precedence-constrained tasks under service-level completion requirements. These coupled factors make participant management central to both time-totarget performance and learning stability. This paper proposes A-CoDa, an adaptive clustered coordination framework for managing dependent FL flows. A-CoDa first uses label-distribution divergence (LDD)-based greedy-balanced clustering to construct statistically coherent and size-aware client groups, which serve as a scalable management abstraction. Building on this structure, we design FedMIX, an uncertainty-aware intra-/inter-cluster participation mechanism that ranks clients by a loss-latency-uncertainty utility and adaptively controls cross-cluster probing according to training progress and latency conditions. A dependency-aware DAG scheduler then orchestrates layer-wise task execution so that parallelism and precedence constraints are jointly respected. We further provide a convergence analysis that frames the result as a sufficient loss-domain design bound, explicitly relating the attainable error floor and sufficient communication rounds to LDDinduced sampling mismatch, residual distribution shift, local-SGD drift, stochastic variance, and adaptive probing budgets. Experiments on handwriting, wearable-sensing, product-image, and medical-imaging tasks evaluate A-CoDa under dependent FL workflows and demonstrate its effectiveness in reducing end-toend completion time while maintaining competitive accuracy.

0.14

How Network Topology and Data Heterogeneity Shape Decentralized Learning: A Spectral Analysis

cs.LG cs.SI ·2026-11-19 ·Luigi Palmieri, Lorenzo Valerio, Chiara Boldrini, Andrea Passarella, Marco Conti +1 more
similarity 0.60
Abstract

Network topology influences decentralized federated learning through information mixing and the relative contribution of individual nodes. While existing analyses primarily focus on spectral mixing properties, they often assume doubly stochastic aggregation, overlooking topology-dependent influence under irregular communication networks. We develop a spectral framework that separates two complementary mechanisms: the dominant eigenmode determines the consensus trajectory and node influence, while non-dominant eigenmodes govern the propagation and attenuation of disagreement. Under uniform neighborhood averaging, we show that the stationary influence is proportional to node degree plus one and derive a bound relating model disagreement to spectral mixing and gradient heterogeneity. We then investigate how these mechanisms interact with the data distribution across nodes. Experiments across regular and heterogeneous network topologies reveal nearly identical learning dynamics under IID data, while network size and connectivity affect how unevenly distributed information is learned under non-IID data. For non-IID settings, we characterize how the distribution of data across nodes shapes the relative contribution of local information to consensus dynamics.

0.16

Communication-Efficient Clustered Federated Learning in Non-IID Wireless Networks

cs.LG ·2026-10-28 ·Martin Isaksson, Rickard Cöster, Šarūnas Girdzijauskas, Seif Haridi
similarity 0.59
Abstract · 64 pages (25 main text + appendices)

Current and emerging mobile networks face increasing privacy, scalability, and performance challenges as distributed learning becomes essential for advanced 6G services. These challenges are amplified by the presence of non-IID data, naturally arising in heterogeneous wireless environments, that degrade the performance of standard Federated Learning (FL) approaches. To address non-IID data across clients, we propose a Clustered Federated Learning (CFL) framework that assigns clients to specialized models during training. Data-domain clustering initializes model-based clustering, enabling identification and exploitation of client similarities without sharing raw data while reducing communication cost. Motivated by mmWave beam selection, we evaluate our methods on a wireless simulation dataset (DeepMIMO) and complement it with two non-IID benchmarks (FEMNIST, Rotated CIFAR-10) to isolate different non-IID characteristics. Our best configurations improve accuracy over IFCA by 4.8 pp on DeepMIMO and 10.6 pp on FEMNIST while reducing downlink data volume by 83.1 % on FEMNIST compared to IFCA. We demonstrate that different clustering strategies suit different non-IID characteristics. These results show that our CFL methods enable resource-efficient, and high-performance distributed learning in future 6G networks.

0.28