AI Agents Learn to Wait, Conserving Resources in Tasks

This research explores a novel concept in sequential decision-making: teaching AI agents to "wait." Traditionally, agents observe and act at every timestep, but many real-world tasks include periods where constant monitoring or control is unnecessary. During these passive phases, an agent could conserve computational resources or redirect its attention to other tasks by simply waiting for the environment to evolve. The authors formalize this "learning to wait" as an optimization problem, aiming to minimize the frequency of sensing and decision-making while maintaining task performance, such as total completion time. They propose training a waiting policy using reinforcement learning, specifically employing lexicographically ordered objectives to prioritize task completion while also minimizing active participation. Experiments conducted across four discrete-state household tasks and three continuous-state environments demonstrate the effectiveness of this approach. The waiting policy successfully learns appropriate waiting behaviors, with agents sometimes pausing for more than half of the task duration. This capability allows pre-trained policies to adapt and incorporate waiting where beneficial, leading to more efficient and resource-aware AI systems.