ReQuant Refines Post-Training Quantization for LLMs, Boosting Accuracy

Post-training quantization (PTQ) is a critical technique for reducing the memory and computational footprint of large language models. Current PTQ methods typically establish an initial quantized model using heuristics or greedy optimization, and these integer assignments are often considered final. This research identifies a gap: the potential for further optimization *after* the initial quantization.The paper introduces ReQuant, a novel fixed-grid discrete refinement procedure designed as a complementary optimization stage for PTQ. ReQuant operates without backpropagation, taking an existing quantized model as a starting point and iteratively adjusting its discrete weight assignments on the fixed quantization grid. Updates are accepted only if they strictly reduce the mean squared reconstruction error and remain within the original grid.This approach transforms the initially fixed PTQ output into an iteratively optimizable discrete solution, making ReQuant a versatile, plug-and-play post-processing stage for any existing PTQ pipeline. Experiments across diverse model families, bit-widths, and downstream tasks show that ReQuant consistently enhances quantized models, yielding significant gains, particularly with simpler initializers and lower bit-widths. Notably, it can refine a basic round-to-nearest initialization to match or exceed the performance of more complex methods like GPTAQ.