Encoding Numeric EHR Data for Transformers: A Comparative Study

This study investigates the optimal methods for encoding numerical data, particularly from Electronic Health Records (EHR), for use with transformer-based AI models. Researchers compared various strategies, including discrete, continuous, and hybrid approaches, evaluating them on both synthetic arithmetic tasks and real-world clinical prediction scenarios. The findings highlight a trade-off between the precision of numerical representation, the stability of model optimization, and architectural flexibility. While methods explicitly modeling value-concept interactions performed best on precision-sensitive tasks, hybrid token-based approaches, which involve binning numerical values before projection, emerged as a more robust and broadly applicable default. The study also noted that models consistently achieve "good enough" numerical computation rather than exact arithmetic, and the clinical benefits of incorporating laboratory values are task-dependent. This suggests that for practical deployment, robustness and ease of use often outweigh the pursuit of maximal numerical precision.