AI Agents Explore Physics Problems, Discovering Statistical Mappings.

A fundamental skill in theoretical physics involves recognizing when a new problem can be mapped onto an already understood model. This study explores whether AI agents powered by large language models (LLMs) can replicate this skill, specifically in discovering statistical mechanical mappings from raw partition functions to more tractable representations. To test this, the researchers developed StatMechBench-v0, a benchmark comprising six Ising-type problems that cover various methods like transfer matrices and gauge-removable disorder. They evaluated a simple "propose-verify-revise" agent using multiple LLMs and different problem phrasings. The findings indicate that numerical feedback is often effective in helping agents correct their code and achieve accurate partition functions. However, a significant limitation was observed: agents could pass numerical checks while still failing to correctly identify the underlying tractable class or accurately assess computational complexity. This suggests that current LLM reasoning has limitations and underscores the necessity for more sophisticated verification methods, such as symbolic checks and structural invariants, beyond mere numerical agreement.