New Kernel Ladder Framework Explores Deep Learning Depth Benefits

The advantages of depth in deep learning models are often debated, with their perceived benefits depending heavily on how the complexity of a representation is defined. This research introduces a novel theoretical framework called the Variation Brownian Kernel Ladder (VBKL) to shed light on this. The VBKL is a path-atomic function-space framework that distinctly separates the process of constructing nonlinear recursive dictionaries from the linear superposition of variations. The framework begins with linear projections, where each "atom" recursively builds unit-ball profiles from the Brownian reproducing kernel Hilbert space. The complete VBKL space is then defined as the signed-measure variation hull of this dictionary. The authors identify each recursive dictionary as a union of Brownian pullback RKHS balls and rigorously establish properties such as variation-controlled Hölder regularity, compactness, and attainment. They also demonstrate strict growth with depth under specific local non-degeneracy conditions. For finite lower-support architectures derived from this framework, the paper provides Rademacher and generalization bounds using concepts like Brownian quadratic chaos and VC entropy. Furthermore, it constructs two-stage approximants, achieving strong error bounds and illustrating the approximation mechanisms through controlled experiments, which suggest a favorable accuracy-complexity trade-off, particularly with limited data.