AlphaEdit Knowledge Editing Limitations Revealed in Reproducibility Study

This paper presents a reproducibility study of AlphaEdit, a knowledge editing method for language models that uses null-space constrained projection to theoretically prevent disruption of previously preserved knowledge. The original AlphaEdit paper by Fang et al. (2025) reported significant gains over existing editing methods on models like LLaMA3, GPT2-XL, and GPT-J. This study successfully replicated AlphaEdit's reported metrics under the original experimental conditions, though it did identify a minor discrepancy in reported fluency and consistency. The researchers extended the evaluation along several axes. When applied to newer model architectures, AlphaEdit's advantages did not generalize uniformly, suggesting that the method relies on architectural assumptions inherent in the "locate-then-edit" paradigm that are not met by these newer models. Furthermore, the study stress-tested AlphaEdit's claim of stable sequential editing by increasing the number of edits far beyond the original paper's scope. It found that while performance was stable at the initially reported scale, it degraded significantly with a much higher edit count, indicating that the null-space projection's protection against catastrophic forgetting is bounded rather than unconditional. Finally, the study expanded the evaluation of edited models to include additional benchmarks such as BoolQ, HellaSwag, and XSTest. The findings revealed that large-scale sequential editing not only impairs general downstream task competence but also negatively impacts safety-relevant refusal behaviors. These results confirm AlphaEdit's performance within its original scope but highlight practical limitations regarding model architecture compatibility and the scalability of its theoretical guarantees.