Execution-Gated Self-Distillation Boosts AI Game Generation Quality

When training code generators, especially for complex tasks like game development, relying on learned judges can lead to models optimizing for proxy features rather than actual functional quality. This means the generated code might score well but still fail to produce a working artifact. This research explores an alternative: using a deterministic, ungameable filter based on actual execution. The study introduces "execution-gated self-distillation," where the primary signal for improvement is whether a generated project successfully launches under a headless engine. This strict-launch criterion acts as a powerful filter. Applying this method to a 14B model (Qwen3-14B+LoRA) for generating Godot projects from natural language briefs, the researchers observed a substantial increase in clean generation rates across four unseen game families. Over three rounds of distillation, the clean generation rate per candidate rose from 8.8% to 42.2%, and best-of-K coverage reached the gold ceiling of 25/25. Crucially, this improvement was directly attributed to the precision of the verifier; a control experiment using a lenient build check instead of the strict launch gate showed no such gains. This highlights that the quality of the verification mechanism directly shapes the model's learning and its ability to produce genuinely functional outputs.