ClinLens Benchmark Challenges Clinical Data Science Agents

Clinical data science agents face the complex challenge of transforming diverse, longitudinal patient records into auditable analyses. However, existing benchmarks often focus on isolated tasks like medical question answering or structured table reasoning, failing to capture the full scope of real-world clinical data science. To address this gap, researchers developed CLINLENS, a comprehensive benchmark comprising 200 executable tasks. These tasks are built upon five linked MIMIC resources, encompassing structured electronic health records, clinical notes, electrocardiograms, chest radiographs, and echocardiograms. The benchmark uses a 4x5 taxonomy, crossing four patient-time scopes with five analysis capabilities, and employs a program-first reverse synthesis approach to ensure rigorous evaluation. Testing 24 standardized model-scaffold configurations on a fixed 126-task suite, the strongest configuration achieved only 56.3% "scope-macro STRICTPASS," despite 100% execution success. Notably, five biomedical systems adapted to GPT-4o-mini reached a maximum of 2.9% "scope-macro STRICTPASS." These results highlight a substantial disparity between current AI model capabilities and the precision required for accurate clinical analyses, underscoring the need for more advanced, long-horizon coding agents in healthcare.