SingularClip Prevents Plasticity Loss in AI Learning

A significant challenge in continual learning and reinforcement learning is the "loss of plasticity," where neural networks, after being trained on a sequence of tasks, lose their ability to adapt to new information or learn subsequent tasks. This phenomenon hinders the development of truly adaptive AI systems. This paper identifies a new root cause for this plasticity loss: the increasing anisotropy, or unevenness, of the singular values of weight matrices during training. The researchers empirically and theoretically analyze this issue, showing how the singular values become increasingly skewed, leading to a reduction in the network's capacity to learn new patterns. To counteract this, they propose SingularClip, a straightforward yet effective procedure. SingularClip involves periodically clipping the singular values of all weight matrices within the neural network. Experiments demonstrate that SingularClip performs strongly against existing baselines across a variety of tasks in both continual supervised learning and deep reinforcement learning. By maintaining a healthier distribution of singular values, SingularClip helps preserve the network's plasticity, allowing it to continue learning and adapting effectively over time without forgetting previous knowledge.