CoVoN Optimizer Improves Variational Continual Learning

Continual learning, where models adapt to new data without forgetting old knowledge, remains a significant challenge for deep networks. Existing optimizers often lack inherent mechanisms to balance stability (retaining old knowledge) and plasticity (learning new knowledge), a balance inspired by neuroscience. This research proposes a solution within the Variational Continual Learning (VCL) framework, where past posteriors serve as priors for future learning. The key innovation is to introduce "slow adaptation" by merging these past posteriors, which helps to slow down knowledge drift over time. This merged posterior then acts as the prior for "fast-weight updates" in the VCL process. These mechanisms are seamlessly integrated into the IVON optimizer, resulting in the new Continual IVON (CoVON) optimizer. CoVoN maintains a similar form and computational cost to Adam but consistently improves upon existing VCL optimizers. It demonstrates superior performance across domain-incremental learning, continual pre-training, and fine-tuning of large language models, offering a more robust approach to lifelong learning for AI systems.