DiG-bench: New Benchmark for AI Discovery in Games

The scientific process heavily relies on discovery—the formulation of novel generalizations through experimentation. However, current AI benchmarks largely overlook this crucial aspect, offering few direct probes into an AI's capacity for discovering new knowledge within controlled environments where objectives are initially unknown. To address this deficiency, researchers have released DiG-bench (Discovery in Games), a new benchmark specifically designed for this purpose. DiG-bench consists of 70 unique games, each encoded as a short string with distinct transformation rules that must be uncovered through interaction and experimentation. Each game presents a series of challenges to verify if the rules have been discovered, with win conditions also initially unknown to the agent. The benchmark offers seven tiers of difficulty, with the lowest tier solvable by multiple models and the highest tier posing a significant challenge even for advanced agentic harnesses. While all 70 games were solvable by humans on their first attempt, only a subset of 21 games is publicly released, with the remainder held private for secure evaluation.