New Benchmark and Memory Framework for Long-Term Embodied AI

A new research paper presents WorldLines, a novel benchmark designed to evaluate embodied AI agents that operate over extended periods in complex, dynamic environments like homes. Unlike existing benchmarks that focus on short-term tasks or language-based memory, WorldLines specifically tests an agent's ability to retain and utilize long-term memory of user routines, world states, and past interactions. The benchmark generates detailed household activity traces, including dialogues, actions, and environmental changes, which are then used for memory-based question answering and embodied task planning. This allows for a more comprehensive assessment of an agent's capacity for sustained assistance. Accompanying the benchmark, the researchers also introduce ObsMem, an observer-grounded memory framework. ObsMem helps agents maintain visibility-aware memories and track state changes relevant to their actions, providing a more robust architecture for making informed decisions in partially observable and evolving environments.