LLMs Fail Information-Seeking in Agentic Clinical Reasoning.

While large language models (LLMs) often achieve impressive scores on medical knowledge assessments, their ability to perform active clinical reasoning, which requires proactively seeking information under uncertainty, remains a significant challenge. A new agentic evaluation framework in hematologic oncology tested models' capacity to request clinical data across three sequential rounds before making a diagnosis and treatment plan. Across 32 frontier models, the highest overall accuracy achieved was only 68%. Information utilization, defined as the fraction of available data actually requested, was the strongest predictor of diagnostic accuracy, with a strong correlation (R = 0.69, P < 0.001). However, utilization dramatically dropped from 57% to 26% in the final round, resulting in critical molecular and cytogenetic data, essential for treatment selection, being overlooked. Although reasoning traces scored high on a clinical reasoning rubric (91% above threshold), they did not correlate well with accuracy, indicating a disconnect between locally coherent rationales and globally correct conclusions. Error analysis revealed that dominant failure modes included search satisficing, anchoring, and premature closure—cognitive biases also characteristic of novice human clinicians. These findings highlight that the primary limitation of current LLMs in clinical oncology is not a lack of medical knowledge but a systematic failure in information-seeking under uncertainty.