Safe-Psych Benchmark Reveals LLMs Struggle with Diagnostic Uncertainty

This paper introduces Safe-Psych, a novel sequential evaluation benchmark specifically designed to assess how large language models (LLMs) perform under conditions of evolving diagnostic uncertainty in clinical psychiatry. While LLMs are increasingly used for healthcare decision support, existing benchmarks often assume complete information is available upfront, failing to test a model's ability to request clarification or abstain when data is insufficient. Safe-Psych comprises over 1,000 real-world psychiatric clinical notes, segmented to simulate the incremental disclosure of evidence. At each stage, psychiatrist-derived action labels (DIAGNOSE, CLARIFY, or ABSTAIN) indicate the appropriate response. The evaluation of state-of-the-art LLMs using this benchmark revealed a significant limitation: even highly capable models struggle with incomplete clinical information. They often exhibit "under-abstention," prematurely diagnosing in over 60% of cases, or shift to "excessive abstention" with safety-aware prompting. The sequential evaluation highlighted that models frequently diagnose before sufficient evidence is gathered and rarely proactively seek clarification. These premature diagnoses were also less accurate. The core issue identified across evaluated models is their difficulty in recognizing when clinical evidence is incomplete and more information is needed. Safe-Psych is released to foster research into improving LLM safety in healthcare.