Beta-VAEs Show Tolerance-Dependent Latent Dimension and Utility Cutoff

Variational Autoencoders (VAEs), particularly $\beta$-VAEs, are known for their ability to learn compressed representations of data. This paper investigates how increasing the regularization strength in a $\beta$-VAE effectively acts as a spectral cutoff, causing latent coordinates with low utility to collapse. In simpler linear Gaussian VAEs, this collapse order perfectly aligns with the reconstruction utility, mirroring a Principal Component Analysis (PCA) spectrum. The study extends this inquiry to fully connected nonlinear VAEs trained on real-world climate data (WorldClim). It reveals that while nonlinear interactions introduce shifts and broaden the onset of this collapse, the fundamental ordering of utility among latent dimensions remains consistent for the resolved ranks. This implies that the spectral cutoff still functions as a utility cutoff, supporting the concept of an "effective description" where only the most useful latent dimensions are retained. The resulting effective-dimension curves highlight a trade-off: increasing the depth of the VAE concentrates utility into a few initial coordinates, improving their fidelity, but simultaneously diminishes the fidelity of the remaining "tail" dimensions. This suggests that the effective dimensionality of the latent space is not fixed but rather depends on the tolerance for information loss.