GameXpert-Bench Evaluates AI Coding Agents for Game Development.

Large language models (LLMs) are increasingly capable of acting as coding agents, generating complete games from natural language descriptions. Game development is a particularly demanding domain, requiring the seamless integration of program logic, visual and audio assets, user interfaces, and playability. Existing benchmarks often fall short by evaluating only the final game product or isolated stages of development. This new research emphasizes the need to assess the full development process. The authors analyzed human-agent game development trajectories and identified three critical stages: initial game generation, bug diagnosis and repair, and iterative optimization. To address this, they introduce GameXpert-Bench, a comprehensive benchmark suite with three tracks corresponding to these stages. GameGen assesses the creation of a complete game from a single request. GameFix evaluates an agent's ability to diagnose and repair reported or self-discovered defects. GameOpt measures cumulative optimization through chains of requests, mimicking real-world iterative development. Each track is evaluated using a combination of live game interaction, deterministic behavioral tests, and final product criteria with regression checks. The suite includes 97 generation tasks, 100 repair tasks with injected bugs, and 17 optimization chains. The findings indicate that current AI agents are more proficient at producing playable foundations and implementing explicit requirements than they are at discovering defects, verifying runtime behavior, or preserving functionality during changes.