Patient-Aware Sampling Improves EHR Foundation Model Performance

This research introduces a novel pretraining technique called "Patient Sampling" for autoregressive foundation models built on Electronic Health Records (EHRs). Traditionally, these models concatenate patient trajectories into a single stream, leading to potential biases where patients with more extensive records disproportionately influence optimization. Patient Sampling addresses this by allowing explicit control over how training signals are distributed among individual patients. This contrasts with the standard "Global Stream" method, which can inadvertently mix data from multiple patients within a single training window and over-represent those with longer histories. Empirical evaluations using real-world EHR data from MIMIC-IV datasets demonstrate that stochastic Patient Sampling, particularly with controllable weighting, significantly enhances model performance. It consistently improves Macro AUROC and AUPRC across various downstream clinical tasks, highlighting the importance of patient-aware sequence construction in developing more accurate and less biased EHR foundation models.