Continual Learning Needs Vary with Environmental Change

The prevailing view of continual learning in large language models (LLMs) often focuses on managing context and preventing forgetting. This paper challenges that perspective, arguing that continual learning is fundamentally about enhancing a model's competence as its environment evolves. The authors propose disentangling environmental change into two axes: spatial (new domains encountered) and temporal (data drift over time for a fixed task). To evaluate methods under these realistic conditions, the researchers recast widely used LLM benchmarks into sequential problems and introduce a mechanism-agnostic protocol. They compare various approaches, including prompt-based methods, supervised learning, reinforcement learning, and context compression techniques. The findings indicate that prompt-based methods adapt quickly but degrade on future tasks, while distillation-based methods accumulate knowledge stably but struggle with updating outdated facts. Context compression improves efficiency but not learning ability. Online reinforcement learning adapts most effectively to knowledge updates but is sensitive to noisy rewards. The core conclusion is that continual learning is not a monolithic capability; different patterns of environmental change necessitate distinct update behaviors, determining when adaptation must be learned within model weights versus through external scaffolding.