Selective Parametric Consolidation Enhances LLM Agent Memory Depth

A new study delves into the concept of "memory depth" for long-running language agents, distinguishing it from mere memory access provided by retrieval systems. While retrieval can fetch past facts, it doesn't ensure that experiences continue to shape an agent's behavior once the working context is no longer active. Memory depth, in contrast, refers to durable, goal-conditioned tendencies written into a small parametric store. To investigate this, researchers developed the "loop-drift protocol," a stress test where retrieval remains intact but working context is unloaded, forcing agents to rely on consolidated memories for persistent goal-conditioned behavior. They propose EVAF, a LoRA consolidation mechanism gated by surprise and valence, which selectively writes important experiences into the agent's parameters. Experiments with GPT-2 and TinyLlama show that while retrieval excels at shallow factual recall, EVAF significantly improves goal persistence and post-unload recovery, achieving high accuracy with minimal parametric writes. Control mechanisms highlight that selective consolidation involves distinct selection and actuation dimensions. The findings suggest that selective parametric consolidation provides a crucial form of memory depth that is distinct from and complementary to traditional retrieval access, though stale-memory invalidation remains an open challenge.