LLMs Generate Synthetic Data for API-Calling Agent Training

Training large language model (LLM) agents that can effectively call APIs typically demands extensive, high-quality datasets of interaction trajectories. A significant hurdle in this process is the necessity for fully functional environments, complete with executable APIs and realistic backend databases, which are costly and time-consuming to develop. This paper introduces an innovative solution that bypasses this requirement by leveraging LLMs themselves as dynamic digital world models. The proposed method generates synthetic data without needing a pre-built environment. It starts with an LLM creating diverse tasks based solely on API specifications. A teacher agent then attempts to solve these tasks, while another LLM acts as a simulator, generating plausible API responses conditioned on the task context and the ongoing simulation history. Finally, an LLM judge filters these generated trajectories to ensure data quality. This approach has been successfully validated on complex benchmarks like AppWorld and OfficeBench, demonstrating that fine-tuning models on this synthetic data significantly boosts performance for API-calling agents, making scalable training feasible across various API ecosystems.