GROM Enables Fast, Gradient-Free Machine Unlearning for LLMs.

Machine unlearning is crucial for removing sensitive or specific information from large language models (LLMs) safely. Current state-of-the-art methods typically involve iterative, gradient-based fine-tuning, which is computationally expensive and may only hide, rather than truly erase, the targeted knowledge. This hidden knowledge can even be recovered through simple techniques like model quantization. A new approach called GROM (Gradient-Free Rapid One-Shot Machine Unlearning) addresses these limitations by offering a direct, exact analytical solution. GROM frames the unlearning process as a ridge-regularized least-squares optimization problem, deriving a closed-form additive update for specific weight matrices. This update forces a selected layer to suppress unwanted content while preserving the model's performance on retained data. Crucially, GROM operates without backpropagation or iterative convergence, relying solely on gradient-free forward passes. This makes it orders of magnitude faster than traditional fine-tuning methods, completing weight edits in mere seconds. Extensive evaluations show GROM achieves state-of-the-art forgetting-utility trade-offs across various benchmarks, significantly reducing computational overhead. Furthermore, because GROM physically removes content from weights, it effectively withstands quantization attacks that can restore content forgotten by gradient-based baselines.