Continual Learning: Modeling Forgetting as Task Interference

A new perspective on continual learning suggests that forgetting should be understood and modeled as direct interference between tasks, moving away from traditional reliance on post-hoc techniques like replay or regularization. The study posits that in scenarios where features are "frozen," forgetting from learning a new task directly corresponds to the interference energy imposed on previously learned tasks. For deep networks, this interference can be quantified through path-averaged curvature with minimal computational overhead. The research highlights that when tasks have disjoint supports, forgetting can be entirely eliminated. Conversely, when task supports overlap in conflicting ways, some level of distortion is unavoidable. Based on this analysis, the authors developed Interference-Gated Functional Allocation (IGFA), a novel method that operates without replay or Fisher information. IGFA intelligently shares model directions when tasks are aligned and actively protects them when conflicts arise. Benchmarking shows IGFA achieves perfect retention for structurally separable tasks and effectively manages the trade-off between plasticity and stability for non-separable tasks, outperforming strong baselines.