FedImp Accelerates Federated Learning Convergence in Non-IID Settings

Federated Learning (FL) offers a privacy-preserving approach to training global models across distributed devices, but it faces a significant hurdle: the non-Independent and Identically Distributed (non-IID) nature of data across these devices. This data heterogeneity often leads to slower convergence and reduced training efficiency. To address this, a new algorithm called Federated Impurity Weighting (FedImp) has been introduced. FedImp's core innovation lies in its ability to quantify each device's contribution to the global model update based on the informational content, or "impurity," of its local data. These contributions are then normalized to compute distinct aggregation weights, ensuring that devices with more informative data have a greater influence on the global model. This dynamic weighting mechanism helps to mitigate the negative effects of non-IID data. Extensive experiments conducted on EMNIST and CIFAR-10 datasets demonstrate FedImp's effectiveness. It significantly accelerates convergence speed, reducing the number of communication rounds by up to 64.4% on EMNIST and 44.2% on CIFAR-10 when compared to established baselines like FedAvg, FedProx, and FedAdp. Furthermore, under highly imbalanced data distributions, FedImp consistently outperformed all baselines, achieving the highest accuracy. This makes FedImp a promising solution for enhancing FL efficiency in real-world, non-IID scenarios.