CliniCARE-Bench Audits Medical Reasoning in EHR for LLM Agents.

While large language models (LLMs) excel on medical knowledge benchmarks, their reliable deployment in clinical settings requires agents capable of defensible investigations using heterogeneous, longitudinal patient records. This involves determining necessary evidence, retrieving and reconciling data, grounding conclusions in verifiable evidence, and knowing when to defer cases. To address this, researchers introduce CliniCARE-Bench, a benchmark for retrospective clinical audit. CliniCARE-Bench comprises 25 clinician-validated scenarios, instantiated as 750 patient-specific cases derived from real MIMIC-IV data. Systems interact with a governed, logged tool environment for data retrieval and policy access, providing one of four verdicts. Beyond simple accuracy, the benchmark scores patient-evidence and policy grounding, process adherence, calibrated abstention, reliability, and efficiency. A key finding is that raw accuracy significantly overstates investigation quality, with defect-free accuracy being substantially lower and reordering leaderboards, highlighting the importance of thorough, auditable reasoning in medical AI.