Decentralized Federated Learning Convergence Slowed by Network Heterogeneities

Decentralized federated learning (DFL) is gaining traction for on-device machine learning model training, offering benefits like privacy preservation, communication efficiency, and resilience against single points of failure. However, the impact of varying structures and time-dependent changes within the communication networks on DFL performance has been largely overlooked. This research delves into these "inhomogeneities" and their effects when model parameters are averaged locally during the aggregation phase. The study establishes a crucial link, demonstrating that the dynamics of DFL, both in its initial stages and its stable long-term behavior, mirror those of a lazy random-walk diffusion process on temporal networks. Based on this mapping, the authors contend that many experimental scenarios in DFL tend to show unrealistically fast convergence because they fail to account for the inherent temporal and structural variations present in actual communication networks. An analysis of real-world temporal networks confirms that these inhomogeneities typically lead to a dramatic slowdown in the diffusion process, and consequently, in the overall convergence of the federated learning system.