Memory Essential for Near-Optimal Generalist AI Agents

The development of generalist AI agents capable of performing near-optimally across a wide range of environments and objectives presents a fundamental question regarding the necessity of memory. This research provides a formal account addressing this very issue. The paper establishes that when two distinct domains share a common observational bottleneck but necessitate different optimal actions, any agent aiming for uniformly near-optimal performance must maintain distinct memory distributions at that bottleneck. This leads to a crucial separation theorem. Furthermore, the research indicates that if an agent's memory contains enough information to estimate values for related goals, it can also approximately reconstruct the agent's local transition dynamics, thereby characterizing memory as vital for domain disambiguation, model reconstruction, and effective planning.