New Benchmark for Evaluating Agent Memory in Streaming Environments

A new benchmark called StreamMemBench has been introduced to assess the memory capabilities of personal AI agents, specifically focusing on their ability to provide future-oriented assistance. Traditional memory benchmarks often test recall or task improvement in isolation, but StreamMemBench simulates real-world scenarios where agents must leverage streaming observations and user interactions over time. The benchmark constructs a two-step task sequence around "evidence anchors" derived from egocentric video streams. The initial task evaluates the agent's use of observed evidence, while the subsequent task measures whether feedback and prior interaction experience are effectively reused. Experiments conducted with various memory systems and backbones revealed that current AI agents frequently struggle to utilize observed evidence or reliably incorporate feedback into future behaviors, even when information is stored. This highlights a critical gap in current agent memory systems for practical, continuous assistance.