Insulin4RL Dataset Enables Realistic Offline RL for ICU Insulin Management

Offline reinforcement learning (ORL) holds significant promise for enhancing clinical decision-making by utilizing historical electronic health record (EHR) data. However, a major challenge in current ORL research and evaluation is the reliance on EHR datasets that have been artificially discretized into fixed, regular time intervals. This discretization creates unrealistic representations of complex clinical scenarios, compromising the generalizability of retrospective model evaluations. To address this, the Insulin4RL dataset has been introduced. This healthcare ORL dataset is unique in that it features naturally irregular inputs and actions, reflecting real clinical trajectories. It is derived from the MIMIC-IV database and comprises over 375,000 labeled decisions from 12,209 patients who required insulin infusion titration in the Intensive Care Unit (ICU). Insulin4RL is designed to support research into ORL model performance under realistic clinical sampling assumptions. The paper provides a detailed description of the dataset's structure and characteristics, along with baseline performance metrics using model-free offline reinforcement learning. A standardized evaluation protocol using fitted Q-evaluation is also included, offering a robust framework for future research in this critical area of healthcare AI.