Noise Explains Grokking Phenomenon in Deep Neural Networks

Deep neural networks (DNNs) exhibit a peculiar phenomenon known as "grokking," where a model suddenly achieves strong generalization performance long after it appeared to have overfit the training data. This delayed onset of generalization has been an open question in the field of deep learning. New research suggests that grokking can be explained by the network's escape from "metastable phases," driven by noise during the training process. The study shows that DNNs undergo first-order phase transitions based on L2 regularization strength, with each transition corresponding to a new learnable feature. Below a critical regularization, multiple features are learnable, but the network can get trapped in low-accuracy metastable states separated by energy barriers. For linear DNNs, the researchers demonstrated that grokking is consistent with hysteresis in these L2 phase transitions. Stochastic Gradient Descent (SGD) noise can provide the necessary impetus for the model to cross these energy barriers, moving from a metastable, low-accuracy state to a generalized state. The escape times follow Arrhenius scaling, and by deliberately trapping models, they reproduced grokking-like delayed convergence. This mechanism likely extends to general nonlinear DNNs, suggesting that task complexity increases the number of metastable states and the potential for hysteresis.