New Method Improves LLM Unlearning by Controlling Support Routes

Enterprises often fine-tune large language models (LLMs) on proprietary data, which may later need to be removed due to privacy, contractual, or compliance requirements. While selective unlearning aims to remove specific knowledge without full retraining, existing methods typically focus only on explicitly identified "forget examples." This approach is insufficient because target knowledge can still be recovered through paraphrases, aliases, or related training examples, leading to persistent data leakage. Researchers have proposed GRAPHSU (Graph-Guided Selective Unlearning), a novel controller that addresses this limitation. GRAPHSU expands the deletion scope by constructing a weighted "support-route graph." This graph propagates deletion pressure beyond the initial forget seeds to high-risk neighboring examples, applying graded forgetting strengths. This ensures that knowledge supported by multiple interconnected data points is effectively removed. Evaluated on benchmarks like the Task of Fictitious Unlearning (TOFU) and PISTOL, using models such as GPT-2 Medium and Llama-3.2-3B-Instruct, GRAPHSU achieved the lowest utility-feasible soft leakage across various deletion settings. It reduced leakage by up to 49.5 percentage points compared to a baseline that only targeted explicit seeds. This demonstrates that effective enterprise unlearning requires a more comprehensive approach that controls the entire network of supporting knowledge, not just isolated data points.