LLM Quantization Transforms: A Comprehensive Survey and New Theory

Modern 4-bit quantization pipelines for Large Language Models (LLMs) commonly begin with a linear transform to optimize outlier mass distribution before rounding. Despite its prevalence, a dedicated survey of this transform stage has been lacking, and its underlying theory has been implicitly re-derived across various works. This research identifies and formalizes a core principle called the "Great Inversion." This principle highlights the fundamental opposition between transform coding, which aims for energy concentration to minimize distortion with flexible bit allocation, and grouped shared-scale quantization, which benefits from within-group flattening due to fixed bit allocation and single group scales. The paper proves this opposition under within-group majorization, showing that the optimal strategies for each objective point in opposite directions. A second critical factor is the number format used for quantization (e.g., FP4, MXFP4, NVFP4), as different formats alter the reward structure for flattening. The study surveys 200 works up to June 2026, classifying 43 transform methods by their structure, data-awareness, and computational cost. It distills a practical guide for selecting transforms based on deployment regimes and outlines key open problems in the field.