How Network Topology and Data Heterogeneity Shape Decentralized Learning: A Spectral Analysis
Abstract
Network topology influences decentralized federated learning through information mixing and the relative contribution of individual nodes. While existing analyses primarily focus on spectral mixing properties, they often assume doubly stochastic aggregation, overlooking topology-dependent influence under irregular communication networks. We develop a spectral framework that separates two complementary mechanisms: the dominant eigenmode determines the consensus trajectory and node influence, while non-dominant eigenmodes govern the propagation and attenuation of disagreement. Under uniform neighborhood averaging, we show that the stationary influence is proportional to node degree plus one and derive a bound relating model disagreement to spectral mixing and gradient heterogeneity. We then investigate how these mechanisms interact with the data distribution across nodes. Experiments across regular and heterogeneous network topologies reveal nearly identical learning dynamics under IID data, while network size and connectivity affect how unevenly distributed information is learned under non-IID data. For non-IID settings, we characterize how the distribution of data across nodes shapes the relative contribution of local information to consensus dynamics.