VQ-bench: A Composable Framework for Vector Quantization Benchmarking

Vector quantization, a long-standing problem, has gained renewed importance in modern AI infrastructure, driving a surge in both engineering and research activities. To address the need for a standardized approach, this paper presents VQ-bench, a unified and open-source framework designed for the development and benchmarking of new quantization algorithms. VQ-bench conceptualizes vector quantization through seven fundamental primitives, demonstrating how these can be arbitrarily composed. The framework successfully re-expresses 25 widely used quantizers as pipelines constructed from these primitives, providing a clear and consistent way to understand and compare different methods. The release of VQ-bench as an open-source tool aims to foster further extension and enable publicly available, reproducible benchmarks. This initiative will help researchers and engineers accelerate innovation in vector quantization by providing a common ground for experimentation and performance evaluation.