AGI Maze Benchmarks LLM World-Modeling in Partially Observable Environments.

Large Language Models (LLMs) excel at pattern completion from static contexts, but their ability to build and manipulate persistent, internal representations of an external world remains a significant challenge. Many tasks that appear as "reasoning" in text become considerably harder when the environment is partially observable, stateful, and demands memory and structured hypotheses about hidden states. To address this, the AGI Maze framework has been introduced. It offers a lightweight, grid-based maze environment designed specifically to benchmark agents that need to learn and utilize world state representations, rather than just inferring local rules from immediate observations. The framework provides a clean API and various difficulty levels. Initial evaluations using AGI Maze reveal that standard LLMs struggle to internally represent mazes during inference. A baseline agent, which leverages its message history as working memory to construct observation descriptions, shows improved performance. However, even this enhanced agent cannot reliably solve small mazes within a human-comparable step budget, highlighting the ongoing difficulty for LLMs in robust world-modeling.