WMLLM: Self-Evolving AI Agents Optimize Black-Box Problems

Black-box optimization problems, characterized by vast and complex search spaces, often struggle with poor sample efficiency due to reliance on direct candidate generation or trial-and-error. This research introduces WMLLM, a self-evolving optimization agent framework that leverages large language models (LLMs) for a "predict-then-act" world modeling approach. The core idea is that LLMs can predict promising optimization directions before costly evaluations, significantly improving search efficiency. WMLLM operates by first predicting potential outcomes and then acting to generate candidates. This process is enhanced through agentic multi-turn refinement, population-based search, and reinforcement learning, allowing the agent to continuously refine both its internal world model and its optimization strategy during the search. Experimental results on black-box optimization tasks, particularly multi-objective molecular optimization, demonstrate that WMLLM achieves state-of-the-art performance and superior sample efficiency under limited evaluation budgets.