FedSPC Improves Personalized Federated Learning by Correcting Shared Parameters.

Personalized federated learning (PFL) is a key approach in federated learning that allows for client-specific model adaptation while handling statistical heterogeneity across devices. Many PFL methods achieve this by dividing the model into shared and personalized parameters, which are then jointly trained on each client's local data. However, a challenge arises because shared parameters are updated by clients optimizing different local objectives, potentially leading to inconsistent updates and weakening the global shared representation. To mitigate this issue, researchers propose Federated Shared Parameter Correction (FedSPC), a modular correction method specifically for PFL. FedSPC applies a control-variate correction exclusively to the shared parameters of a given PFL method, leaving the personalized parameters untouched. This targeted correction helps to harmonize the shared updates. FedSPC is designed to be integrated into various common PFL settings, including those with shared feature extractors, shared classifiers, or fully shared models with local regularization. Experimental results on datasets like CIFAR-100 and Tiny-ImageNet, using models such as ViT, ResNet-34, and VGG-11, demonstrate that FedSPC consistently improves performance across several representative PFL methods, including FedPer, FedRep, FedBABU, LG-FedAvg, and Ditto.