Grid-Based ANN Search Shows Robust Scaling in High Dimensions

This research provides a detailed analysis of the scaling behavior of a multiprobe grid algorithm for Approximate Nearest Neighbor (ANN) search, specifically examining its performance with respect to dataset size (N) and dimensionality (d). Grid-based ANN approaches have historically been overlooked in modern scaling analyses, making this study particularly relevant. The experiments uncovered a previously unobserved dimensional scaling crossover when applied to the GloVe embedding family. Unlike graph-, tree-, and partitioning-based methods that show degrading throughput in high dimensions, the multiprobe grid search maintained an approximately constant dimensional scaling exponent. This advantage is coupled with near-linear query scaling in N and significantly lower indexing costs compared to competing ANN methods. The findings suggest that grid-based methods, such as multiprobe grid, could be highly competitive in scenarios demanding frequent index rebuilds or involving very high-dimensional data, where indexing cost and dimensional robustness are critical performance factors. The study also notes the relevance of ANN scaling properties for understanding efficient transformer architectures.