AI Improves Cost-Aware Sequential Medical Diagnosis with Novel Reinforcement Learning.

This research introduces CDPR (Counterfactual Diagnostic Process Reward), an innovative reinforcement learning approach designed to optimize sequential medical diagnosis. Unlike traditional AI models that treat diagnosis as a single-pass classification, CDPR simulates the step-by-step, cost-aware process a physician follows, ordering tests and updating diagnoses iteratively. The core challenge addressed is credit assignment in long diagnostic trajectories, where a wasteful process might receive the same final score as an efficient one. CDPR overcomes this by identifying points of policy hesitation and scoring chosen actions based on their advantage over alternatives, using short rollouts that balance correctness with test count and cost. This method requires no expert labels or learned critic, making it more practical. Integrated into GRPO, CDPR was tested on multiple datasets, including MIMIC-IV and a private hospital dataset. The results demonstrate that this framework not only enhances diagnostic accuracy but also substantially reduces the number and associated costs of medical examinations. By focusing on the trade-offs between test value and cost, CDPR offers a more realistic and efficient AI solution for clinical decision-making.