Memory-Augmented Speculation Accelerates LLM Agents Losslessly.

Speculative execution is a technique used to speed up Large Language Model (LLM) agents by employing a smaller, more efficient model to predict and pre-launch the next action while the primary environment is idle. However, existing speculative methods are typically stateless, discarding valuable information between tasks and thus limiting their ability to improve over time. This research introduces a significant enhancement by integrating three online memory systems into the speculator model. These systems include a contrastive transition table that tracks action-sequence statistics, an episodic memory for retrieving contextually similar past segments, and a confusion tracker to suppress recurring errors. This memory augmentation allows the speculator to learn from past agent trajectories, continuously improving its prediction quality. Evaluations across six benchmarks and three types of speculation (action, observation, and chained prediction) demonstrated substantial gains. Memory-augmented speculation led to a 19-39% relative accuracy improvement in action prediction and up to a 2.5x increase in observation prediction, particularly in tasks with repetitive action spaces. These benefits grow with accumulated experience and are applicable across various speculator models. Crucially, this acceleration is "lossless" as it occurs during idle time, incurring no additional wall-clock cost and ensuring the main agent's trajectory remains identical to non-speculative execution.