Random Projections May Not Preserve Key Geometric Information.

A new study challenges the common assumption that random projections, as guaranteed by the Johnson-Lindenstrauss (JL) lemma, adequately preserve all essential geometric information in high-dimensional data. While the JL lemma ensures that pairwise squared distances are maintained within a certain error margin after projection to a lower dimension, this research demonstrates that such preservation can be misleading. For instance, a random replacement map can satisfy the JL bound even if the projected data is entirely independent of the original, rendering the bound uninformative about retained geometry. The paper delves into how well any decoder can recover features of squared distances from a linear sketch. For isotropic Gaussian data, the authors diagonalize the recovery operator, providing a closed-form solution for its singular values. This analysis leads to three sharp consequences: a rank-m sketch retains at most an m/d fraction of the variance of any single squared distance feature; expected Kendall correlation vanishes when log n << m << d, even if the JL bound is satisfied; and Haar-averaged retained covariance-shape information is only (m/d)^2. These findings indicate that JL distance preservation does not necessarily quantify the geometry available for comparison or inference, highlighting a significant limitation for tasks like nearest-neighbor search or clustering.