New Local Learning Architecture Reduces Catastrophic Forgetting in AI

Catastrophic forgetting remains a major hurdle in continual learning, where AI models tend to forget previously learned information when trained on new tasks. Traditional methods often rely on end-to-end backpropagation, which can lead to instability and rapid degradation of performance on older tasks. This paper presents the Cognitive Memory Primitive (CMP), a novel architecture designed to mitigate this problem. CMP represents inputs as sparse relational codes and stores them in a two-tier competitive memory. Crucially, it learns through local updates, bypassing the need for end-to-end backpropagation through its feature-generating system. Experiments on a byte-level language modeling protocol show that CMP exhibits substantially lower backward transfer (i.e., less forgetting) compared to a parameter-matched Transformer using online Elastic Weight Consolidation. While CMP shows promise in reducing forgetting, the authors transparently report a significant accuracy gap on single-domain tasks and a null result on a vision benchmark, emphasizing the trade-offs involved.