Hierarchical ICD Code Modeling Improves EHR Foundation Models

A recent study explores how to improve Electronic Health Record (EHR) foundation models by leveraging the inherent hierarchical structure of ICD diagnosis codes. Traditionally, these models treat ICD codes as independent, flat data points, overlooking the rich clinical relationships embedded within their hierarchical organization, which defines disease families and subcategories. The researchers investigated two primary methods for integrating this hierarchy: augmenting diagnosis sequences in transformer models with hierarchical tokens and injecting hierarchy into graph-based code representations using specialized edges. These methods were evaluated on large clinical datasets, MIMIC-IV and eICU, to assess their impact on predictive accuracy and transferability. The findings indicate that explicitly encoding ICD hierarchy consistently enhances model performance, both within the same dataset and when transferring knowledge to new datasets. The optimal level of hierarchy to incorporate was found to vary depending on the specific task and modeling technique, confirming the generalizable benefits of this hierarchy-aware approach for clinical representation learning.