New Benchmark Evaluates Prospective Memory in LLM Agents.

A critical challenge for AI agents is prospective memory, which is the ability to remember and execute a planned action at a specific future time or when a particular condition arises, even while engaged in other activities. To address this, a new benchmark called PM-Bench has been developed. This benchmark is text-based and draws inspiration from the "Virtual Week" paradigm used in cognitive science. PM-Bench simulates a seven-day week where LLM agents must continuously perform an ongoing task while simultaneously monitoring for cues to execute deferred intentions. The evaluation of eight leading LLMs across various configurations showed that prospective memory remains a significant hurdle. The top-performing agent, a GPT-5.4 variant, only achieved a 65.1% F1 score, indicating substantial room for improvement. The study also noted that no single strategy consistently improved prospective memory across all models.