Explainable AI for Electronic Health Records Foundation Models.

Foundation Models for Electronic Health Records (FEMRs) are powerful tools that convert complex patient data into generalizable representations for various clinical predictions. However, their "black-box" nature raises significant concerns regarding bias, interpretability, and trust among clinicians. This research addresses these issues by proposing X-FEMR, the first approach to provide token-level explanations for FEMRs. X-FEMR works by training a Transformer-based surrogate model. This surrogate learns to mimic the input-output behavior of the FEMR across different prediction tasks, crucially preserving the temporal dynamics inherent in patient trajectories. By analyzing this surrogate, the method identifies the most influential tokens (e.g., specific diagnoses, medications, or lab results) that contribute to the FEMR's predictions, offering insights into how the model processes patient history. To validate the clinical relevance of these explanations, the researchers developed a novel clinical alignment metric. This metric quantifies how well the surrogate model's key tokens correspond to features already validated by clinical experts. Results show that the surrogate accurately approximates FEMR predictions and that its token-level explanations align well with clinical knowledge, providing a practical framework for building more interpretable and trustworthy AI in healthcare.