TRACE Improves LLM Safety Post-Fine-Tuning Without Utility Loss

Fine-tuning Large Language Models (LLMs) for specific tasks, often through Fine-Tuning-as-a-Service (FTaaS) platforms, can inadvertently compromise their safety alignment. Service providers face the challenge of restoring safety without re-running full alignment or losing the utility gained from customization. Existing parameter merging methods, which add a "safety patch," often struggle with task-safety update entanglement, where safety and task updates conflict, making calibration difficult. This paper proposes TRACE (Trajectory-Based Safety Patch Learning), shifting the focus from online merging operators to offline patch learning. TRACE aims to find a safety patch that effectively controls unsafe behaviors while minimally interfering with task-relevant directions. It achieves this by simulating harmful tuning trajectories to generate progressively corrupted model states. The framework then optimizes a plug-in patch designed to recover safety and maintain utility across these varying corrupted states. Evaluated across six benchmarks and two different models, TRACE consistently outperformed other methods, achieving nearly 100% safety while preserving utility comparable to the undefended fine-tuned model.