Understanding Neural Network Optimization and Feature Learning.

Despite the widespread deployment and impact of neural networks, the fundamental mechanisms by which they learn and arrive at their solutions remain poorly understood. This lack of understanding is critical, especially as AI systems make decisions in sensitive domains like healthcare and autonomous driving, where unintended consequences can be severe. This dissertation tackles this challenge by investigating the optimization process, or "training," of neural networks. The core of the research delves into the "loss landscape," the complex surface that guides the network's learning. A particular focus is placed on "mode connectivity," a curious phenomenon where different optimal solutions (modes) in the loss landscape can be connected by paths of low loss. Explaining this structure is key to understanding how neural networks learn diverse features. By elucidating, explaining, and exploiting this special structure, the dissertation aims to provide deeper insights into how neural networks acquire and represent features. This foundational understanding is crucial for developing more reliable, interpretable, and robust AI systems, moving beyond empirical successes to a more principled approach to neural network design and deployment.