LLMs Struggle with Real Scientific Discovery, New Benchmark Shows

A new benchmark called Science Edge Evaluation (SEE) has been introduced to assess the capability of large language models (LLMs) in supporting complex, real-world scientific discovery. This multimodal benchmark features expert-curated questions derived from peer-reviewed literature and experimental practices in chemistry, biology, and materials science. The evaluation of 19 different multimodal LLMs (MLLMs) showed that even the best-performing model achieved only 48.7% accuracy, indicating a significant limitation. Interestingly, general-purpose MLLMs often outperformed science-specialized models. While tool use improved accuracy slightly to 52.7% in visual-agent evaluations, it did not fundamentally solve the core challenge: MLLMs struggle to manage tool-derived information within the bounds of original experimental evidence. The findings suggest that current MLLMs are not yet capable of reliably making justified, evidence-bounded inferences from experimental results, a crucial step for genuine scientific discovery, and need to evolve from explaining established concepts to deriving novel insights.