New Method Improves Continual Learning Without Replay Data

Researchers have developed a new approach to continual learning (CL) for deep neural networks, focusing on scenarios where access to previous task data (replay) is unavailable and the system cannot identify the task at inference time. The core innovation is a measure called "relative parameter-importance," which quantifies how crucial each network parameter is for both the current learning task and previously learned tasks. Parameters deemed highly important for past tasks are heavily regularized to maintain stability, preventing "catastrophic forgetting." Conversely, parameters with low relative importance are allowed more freedom to update, facilitating the learning of new tasks and, uniquely, enabling backward knowledge transfer. This selective regularization strategy helps balance the stability-plasticity dilemma inherent in continual learning. The method demonstrates improved performance over state-of-the-art CL techniques in both class-incremental and domain-incremental text classification problems, with potential for extension to text generation.