New Method Enables Continual Unlearning for Multimodal LLMs

A novel method called Merging for Continual Unlearning (MCU) has been developed to enhance the unlearning capabilities of Multimodal Large Language Models (MLLMs). Traditional MLLM unlearning techniques are typically designed for one-shot requests, leading to issues like cumulative utility loss and "unlearning rebound" when applied repeatedly in continuous scenarios. MCU tackles this by dynamically merging multiple individual unlearning adapters into a single, unified adapter each time a new unlearning request is made. The core of MCU lies in its ability to manage the complex cross-task dependencies among these unlearning adapters. While these dependencies can facilitate knowledge transfer, they also risk introducing interference that compromises both unlearning effectiveness and the preservation of retained knowledge. MCU addresses this by projecting adapters into a shared space, preserving dominant directions, suppressing over-concentrated coordinates, and reconfiguring dependencies to minimize interference while maximizing transferability. Experimental results on benchmarks like ICU-Bench and MLLMU-Bench demonstrate MCU's superior performance in unlearning while maintaining model utility.