Statistical Mechanics Explains Machine Learning and Memorization

This doctoral thesis delves into the theoretical underpinnings of artificial neural networks (NNs) and machine learning (ML) algorithms, aiming to address current gaps in understanding that hinder their full potential. A primary focus is on the phenomenon of adversarial attacks, where NNs are easily misled, and the nature of implicitly low-dimensional learning structures that guide NN weight adjustments during training. To achieve this, the research employs mathematical tools from statistical mechanics. It examines how different types of NNs, specifically dense associative memory (DAM) and restricted Boltzmann machines (RBM), learn and memorize data. By studying these models and their interconnections, the thesis seeks to provide a clearer theoretical framework for understanding ML behavior, particularly regarding how models fit data with varying degrees of learning versus memorization, and to shed light on the vulnerabilities exploited by adversarial attacks.