LLM Agents Conduct Controlled Experiments with Simulation Models

While Large Language Models (LLMs) excel at reasoning, planning, and tool use, many scientific and engineering tasks demand an understanding of system responses to interventions, which typically requires controlled experimentation. This research introduces a multi-agent framework that empowers LLM agents to conduct such experiments using scientific simulation models. The initial application focuses on pharmaceutical process design. The proposed system operates by taking a user query and a baseline configuration, then constructing a structured task representation. It designs experiments, executes comparative simulations, interprets the outcomes, and synthesizes evidence-based recommendations for process parameter optimization. By integrating LLMs with high-fidelity simulation models within an interactive agent framework, the system facilitates reasoning through intervention, comparison, and observation. This integration results in outputs that are significantly more specific and actionable compared to those generated by language-only reasoning. In an industrial application setting, this advantage translates to higher output specificity and improved user ratings for correctness and helpfulness. Ablation studies and case analyses further confirm the effectiveness and practical utility of this simulation-integrated experimental reasoning approach.