New Hyperbolic Networks Improve Visual Representation Learning

Recent advancements in machine learning have explored learning visual representations directly within hyperbolic space, as seen with models like the Poincar\'e ResNet. However, optimizing these hyperbolic networks presents significant hurdles. These include the high computational cost associated with Riemannian gradients and the strict boundary constraints inherent to the hyperbolic manifold, which can impede effective learning. Furthermore, traditional hyperbolic networks often treat different spatial transformations of the same object as distinct concepts, leading to inefficient parameter usage and potential signal degradation. To overcome these limitations, a new approach called Equivariant Poincar\'e ResNets has been introduced. This method ingeniously combines hyperbolic geometry with discrete symmetry groups, specifically C4 and D4. The researchers identified and addressed critical roadblocks in adapting Euclidean equivariance to hyperbolic space. Key innovations include geometrically safe tensor reshaping, the use of left-regular permutations for hyperbolic group convolutions, and a novel joint-orientation Poincar\'e Midpoint Batch normalisation. Empirical results demonstrate that embedding equivariance into these hyperbolic networks drastically reduces the complexity of the optimization space. This not only accelerates convergence significantly but also ensures that the boundary constraints of the Poincar\'e ball are respected, all while preserving crucial spatial-group equivariance. This advancement promises more efficient and robust learning of visual features in hyperbolic domains.