RDQ Quantization Boosts LLM Performance Below 4-Bit Precision

Quantizing large language models (LLMs) after training often leads to a sharp drop in performance when precision falls below 4-bit. This research identifies the primary cause as "residual stream distributional drift," where quantization noise accumulates across transformer layers, causing the residual representation to diverge significantly from its full-precision baseline. The study found that a large percentage of LLaMA-3-8B layers exhibit non-Gaussian residual distributions, with variance growing exponentially with depth. To combat this, the paper introduces RDQ (Residual Distribution Quantization), a post-training quantization (PTQ) framework centered on Cascaded Error Compensation (CEC). CEC is a sequential calibration process that accurately captures the drifted activations each layer receives by running calibration data through already-quantized upstream layers. It then fits per-channel AWQ-style scales against these drifted inputs. These scales are mathematically folded into the preceding RMSNorm weights, ensuring exact equivalence without any additional inference overhead. RDQ achieves state-of-the-art perplexity scores across LLaMA-3-8B, Qwen-2.5-7B, and Mistral-7B models at 3-bit and 4-bit weights, outperforming existing baselines by substantial margins. The output is standard group-128 asymmetric quantization, compatible with common inference stacks.