Thesis Explores Bayesian Principles for Deep Learning Uncertainty and Generalization

This doctoral thesis presents a comprehensive investigation into applying Bayesian principles to deepen the understanding of modern deep learning systems. The core focus is on two critical aspects: the ability of neural networks to generalize effectively and their capacity to quantify uncertainty in their predictions. The research approaches these challenges from both methodological and theoretical perspectives, unifying Bayesian inference, function-space modeling, and large-deviation theory. Methodologically, the thesis introduces the Deep Variational Implicit Process (DVIP), a scalable Bayesian framework that extends implicit processes to deep architectures. Additionally, it proposes two post-hoc methods, the Variational Linearized Laplace Approximation (VaLLA) and the Fixed-Mean Gaussian Process (FMGP), designed to equip pre-trained deterministic networks with well-calibrated uncertainty estimates without requiring retraining. The theoretical contributions address the fundamental question of why large, over-parameterized neural networks generalize so effectively. The thesis develops a unified probabilistic framework that elucidates the connections between three key mechanisms—diversity, smoothness, and stochasticity—within the context of PAC-Bayesian and large-deviation theory. This work provides a more profound theoretical foundation for understanding deep learning's generalization capabilities.