New Theory Explains Neural Network Generalization Beyond Overfitting

Modern neural networks often exhibit a surprising ability to generalize well, even when they have more parameters than training data points, a phenomenon known as "double descent." This paper introduces a novel theoretical explanation for this behavior. It posits that the generalization capability arises from a phase transition occurring during the network's training. This phase transition is characterized by a breakdown of the fluctuation-dissipation theorem and a state of broken ergodicity within the stochastic field theory that describes the training process. The researchers draw a parallel between the network's rigidity in generalization and the London model of superconductivity, suggesting a fundamental physical analogy for how these complex systems learn and perform.