Fisher Width: A New Geometric Measure for Statistical Model Complexity

Researchers have developed a new mathematical concept called Fisher width, designed to measure the complexity of statistical models. This measure is a counterpart to Gaussian width, which is commonly used in Euclidean geometry to understand the "size" or "extent" of high-dimensional sets. However, statistical models operate within a different kind of geometric space, known as statistical manifolds, where distances are defined by how distinguishable different statistical parameters are, rather than simple Euclidean distances. Fisher width adapts this concept by incorporating the Fisher information metric, which inherently accounts for the unique curvature and scaling of statistical manifolds. This makes the measure robust to how a model's parameters are represented and sensitive to the underlying statistical properties. The paper outlines the theoretical foundations of Fisher width, demonstrating its consistency with Gaussian width's properties while also capturing unique anisotropic effects. As a practical application, the authors show how Fisher width can be used to derive generalization bounds for certain types of machine learning models and propose methods for estimating it. Empirical tests on the MNIST dataset across various model architectures confirm its utility in understanding model complexity in a statistically meaningful way.