New Framework Unifies Task Detection and Adaptation for Continual Learning

Mitigating catastrophic forgetting is a major challenge in continual learning (CL) for large language models (LLMs), especially when new tasks arrive sequentially. Existing methods typically rely on explicit task boundaries to constrain parameter updates or introduce task-specific adaptation modules, which limits their applicability in realistic "task-free" scenarios where such boundaries are unknown. Researchers have introduced FiUni (Fisher-guided unified), a novel framework designed for batch-level task detection and parameter-efficient continual adaptation. FiUni is motivated by a key observation: the orthogonality of principal subspaces within the Kronecker-Factored Approximate Curvature (K-FAC) approximation of the Fisher Information Matrix (FIM), estimated from small samples, can indicate task similarity. Leveraging this insight, FiUni constructs FIM-derived frozen subspaces to guide low-rank adaptation (LoRA). It dynamically matches the Fisher principal subspace of each incoming batch window with historical subspaces, allowing the system to adaptively decide whether to reuse existing knowledge, expand a related subspace, or create an entirely new one. This approach effectively balances knowledge sharing and task isolation, demonstrating competitive performance against advanced task-aware CL methods with fewer trainable parameters.