New Framework Improves Private Model Merging Efficiency

Model merging offers a way to combine independently fine-tuned task models into a single multi-task model without needing access to the original training data. This is particularly appealing when data cannot be centralized. However, when these task models are trained with differential privacy (DP) to limit data leakage, their mergeability can be significantly compromised. This study investigates the geometric challenges of merging differentially private models. The research identifies two key geometric obstacles: "local sharpness," which makes task losses overly sensitive to parameter changes during merging, and "reference drift," which describes how private task models diverge from their shared pretrained initialization, amplifying cross-task interference. These factors make private models difficult to combine effectively. To address these issues, the authors propose DP-Merging, a geometry-aware framework. DP-Merging uses a DP-compatible sharpness-aware objective to guide private task models towards flatter loss regions and incorporates a reference-based alignment regularizer to keep them closer to the initial pretrained state. Experiments across vision and language tasks demonstrate that DP-Merging consistently improves the performance of privately merged models while maintaining the underlying differential privacy guarantees.