LieBN Introduces Batch Normalization for Lie Group Manifolds

This paper presents LieBN, a novel framework for performing Batch Normalization (BN) on data that resides on Lie groups, a type of manifold. In many machine learning applications, measurements are manifold-valued, and while deep neural networks (DNNs) have been extended to operate on these geometries, effective normalization techniques, often called Riemannian normalization, have been limited. Existing methods are either specific to certain manifolds or struggle to normalize manifold-valued sample distributions effectively. LieBN overcomes these limitations by leveraging the theoretically robust left- and right-invariant metrics inherent to every Lie group. This approach provides strong theoretical guarantees for controlling both the Riemannian mean and variance of the data. The researchers demonstrate the instantiation of LieBN across nine different geometries, including four on the Symmetric Positive Definite (SPD) manifold, one on the group of rotation matrices, and four on the manifold of full-rank correlation matrices. Notably, the framework introduces a new right-invariant metric for SPD manifolds and extends three existing Lie group structures through matrix power deformation. Extensive experiments across these diverse manifolds validate the effectiveness of LieBN in normalizing complex, manifold-valued data, paving the way for more stable and performant DNNs in these specialized domains.