Gravitational Theory Explains AI Fine-Tuning Reversion

AI models sometimes exhibit a phenomenon where fine-tuning on benign data can unexpectedly undo previously learned behaviors, including safety alignments. This can lead to the re-emergence of unlearned capabilities or the transfer of latent traits through seemingly unrelated updates. This paper introduces a 'gravitational interpretation' to explain these fine-tuning reversion phenomena. The hypothesis posits that large, early training phases establish dominant behavioral patterns or 'manifolds' within the model's architecture. Subsequent alignment or specialization phases are viewed as relatively shallow adjustments or displacements from these deeply ingrained patterns. Consequently, later fine-tuning can inherit a persistent 'reversion component' that pulls the model back towards these dominant, earlier-formed manifolds. The research demonstrates that blocking motion along this identified reversion direction can significantly reduce harmfulness while maintaining task performance, supporting its role as a causal mediator of early post-alignment reversion.