LLMs Struggle to Generate Executable Unity Game Code in Single Pass

Research investigated the capability of large language models (LLMs) to autonomously generate functional Unity C# code for game scenes. Unlike typical demonstrations that rely on iterative repair loops to fix errors, this study focused on a stringent single-pass generation, where the initial code draft was considered final. This approach aimed to isolate and assess the LLM's inherent parametric knowledge.The experiment involved 10,400 generations across four open-weight LLMs (7B-30B parameters) and various conditioning levels, targeting 26 distinct "Goal Playable Concepts." The findings were stark: none of the generated C# scripts compiled into a runnable scene. A detailed census of 90,673 compiler errors categorized them into "Grounding" (invented or misused Unity types/APIs) and "Hygiene" (structural C# defects).The distribution of these error types varied significantly by the game concept, with larger models and stricter intermediate representations shifting error patterns but never achieving a compiling scene. The primary bottleneck identified was the LLMs' deficiency in engine-specific knowledge, highlighting a critical limitation for single-pass code generation in complex environments like Unity.