New Framework Analyzes Forgetting in Continual Learning.

Continual learning systems must acquire new knowledge without forgetting previously learned tasks, a challenge known as catastrophic forgetting. Replay-based methods, which mix a small buffer of old examples with new data, are effective but their generalization behavior is complex. This paper proposes a novel information-theoretic framework that dissects the problem at each layer of a neural network. The framework separates the generalization gap into two distinct components: "replay-induced representation drift," which arises from using a finite memory buffer as a proxy for past data distributions, and an "optimization-dependence term," which captures how the buffer, current data, and the final model interact during training. The optimization term is further broken down into stability, plasticity, interaction, and residual-coupling elements. The researchers introduce practical refinements, including a Wasserstein relaxation for the drift term to handle support mismatch, which helps identify the optimal internal layer for stabilization. An SGLD instantiation of the optimization term provides a trajectory-level budget and a curvature-aware gradient-alignment statistic, serving as an online diagnostic for task-wise forgetting. Experiments confirm the framework's predictions regarding memory scaling, internal layer dynamics, and the alignment signal's link to forgetting.