Fair, Personalized Decentralized Learning Achieves Communication Efficiency

Decentralized learning systems allow multiple clients to collaboratively train models without a central coordinator, offering benefits in scalability, privacy, and robustness. However, these systems face significant challenges, including statistical heterogeneity among clients, ensuring fairness in individual client performance, and managing stringent communication constraints. Researchers have introduced DMFL-SQ, a unified framework designed to address these challenges simultaneously. This decentralized multi-task learning algorithm integrates three key components: graph-based personalization, which tailors models to individual clients while leveraging network structure; an agnostic mixture fairness objective, ensuring equitable performance across diverse client groups; and compressed event-triggered communication, which drastically reduces data exchange through sparsification, quantization, and asynchronous updates. The framework provides strong theoretical convergence guarantees for general non-convex objectives, achieving an optimal rate despite the sparse, quantized, and event-triggered communication. Additionally, it offers PAC-Bayes generalization guarantees for its fairness-aware objective. Empirical evaluations on datasets like CIFAR-10 and the MUSMET EEG dataset confirm that DMFL-SQ substantially cuts communication overhead while preserving predictive performance and significantly enhancing fairness across clients.