Federated Learning Improves EHR Foundation Models Across Health Systems

This study explores the potential of federated learning for training generative event models (GEMs) on tokenized electronic health records (EHRs) across multiple, institutionally siloed health systems. The research utilized data from over 122,000 intensive care hospitalizations across three independent health systems, harmonized to a common data format. The evaluation focused on 12 clinical prediction tasks and compared within-site, cross-site, centralized, and federated training configurations. GEMs demonstrated superior mean within-site and cross-site performance, exhibiting significantly better transportability than conventional supervised models like LightGBM. Federated Learning (FedAvg and FedAvgM) effectively approached the performance of centralized training, with most benefits realized within a few communication rounds. The findings highlight that while federated GEM training is feasible and effective, the primary challenge remains learning truly transportable representations that consistently benefit target sites with heterogeneous data.