Weight Magnitude and Direction Play Dissociable Roles in Grokking

This paper delves into the mechanisms behind "grokking," the phenomenon where neural networks generalize long after achieving high training accuracy. The authors introduce a novel experimental technique called "cross-trajectory chimera interventions." This method involves taking two independently trained networks, splitting their weight vectors into norm (magnitude) and unit direction, and then recombining one run's norm with another's direction before continuing training. Through these interventions, the study reveals that weight direction carries a distinct, transferable circuit identity. Implanting a donor network's weight direction into a recipient network, while using the recipient's weight norm, consistently drives the recipient to adopt the donor's specific circuit solution. In contrast, weight magnitude primarily exerts a modest, distributed delay effect and does not convey any identity signal. The transfer of circuit identity is found to be threshold-like, predicted by the recipient's norm, indicating that norm governs how easily a learned identity can be overwritten.