DOW-KE Improves Multi-Layer Knowledge Editing in LLMs

Existing methods for editing knowledge within large language models (LLMs) often follow a "locate-then-edit" pipeline. This typically involves first optimizing intermediate representations, known as "anchors," at specific layers, and then translating these into layer-by-layer weight updates. A key drawback of this approach is that the joint effect of these multi-layer weight updates, as they propagate through the model's true forward pass, is never directly optimized. This creates a "closure gap" where local optimizations fail to fully account for propagation-induced attenuation and distortion. The new method, DOW-KE (Direct End-to-End Weight Optimization for Knowledge Editing), addresses this by adhering to a single principle: what is optimized must be exactly what is deployed. DOW-KE backpropagates the final editing objective through the entire model, allowing for the joint optimization of weight updates across all edited layers. This ensures that cross-layer propagation and coupling are considered in every gradient step, closing the aforementioned gap. Furthermore, DOW-KE embeds the preservation projection directly into the update parameterization within the computation graph. This means every gradient acts on the deployed update, preventing post-hoc constraints from reopening the gap and ensuring edits remain distinct from protected knowledge. Extensive experiments on large-scale sequential editing tasks across various datasets and models show that DOW-KE consistently achieves superior overall scores and neighborhood specificity compared to evaluated baselines.