LLMs Achieve Continual Scientific Discovery with Evolving Beliefs

Large Language Models (LLMs) are increasingly used in scientific discovery, often in a loop of hypothesis generation and verification guided by a "Bayesian surprise" metric. However, existing approaches treat this surprisal as a static value, failing to account for evolving beliefs as new evidence emerges. This research addresses this limitation by proposing "evidence-informed LLM beliefs." The new framework allows LLMs to update their prior beliefs with evidence from previously tested hypotheses, enabling the computation of non-stationary surprisal. This dynamic approach better mimics human reasoning. The study found that retrieval-augmented generation based on embedding-based retrieval of prior discoveries is effective for in-context belief updating, identifying and filtering out spurious surprisals. By modifying the hypothesis search procedure to avoid these spurious rewards and prioritize diverse, genuinely surprising hypotheses, the method significantly boosts accumulated non-stationary surprisal across various discovery domains. This demonstrates that true continual scientific discovery with LLMs requires both adaptive belief measurement and intelligent search strategies that prevent redundancy.