K"ahler Geometry Explores Complex Neural Network Landscapes.

This research delves into the intricate optimization landscapes of neural networks that utilize complex-valued parameters. The approach is grounded in an information-theoretic manifold perspective, applying classical optimization guarantees adapted for complex geometric varieties, particularly through Dolbeault asymptotics. The study posits that the descent path in such networks admits a K"ahler information metric when using cross-entropy, derived from the Wirtinger Hessian on the log-likelihood potential. By restricting to a natural gradient descent update rule, the descent path is maintained within the holomorphic tangent bundle. A significant focus is placed on Calabi-Yau information manifolds, which are shown to undermine theoretical guarantees due to ill-curvature-conditioned landscapes. The paper further explores how a fixed determinant under a Calabi-Yau metric, especially in a non-compact setting, can lead to a "blow-up" effect if the metric is almost low rank. It also expands on the discovery that negative curvature, specifically sectional curvature, can subvert the loss landscape, drawing connections to failure modes of neural network guarantees under vanishing and negative Ricci curvature. The arguments are primarily geometric analytic, linking deep learning theory to initialization asymptotics and curvature-related failure modes.