Signed Symmetric Quantization Boosts LLM Efficiency at Low Bits

This paper introduces "signed symmetric quantization," a novel approach to quantizing large language models (LLMs) using few-bit integers. The core problem it addresses is the inherent asymmetry in signed integer alphabets, which contain one more negative value than positive. Conventional symmetric quantizers typically fix their scale to be positive, leading to potential clipping of positive outliers and increased quantization error, especially at very low bit precisions. While asymmetric quantization can mitigate this by introducing a zero point, it often incurs a significant runtime performance penalty. For instance, in specific CPU environments, asymmetric 4-bit formats can use more memory and have substantially lower throughput compared to symmetric counterparts. The proposed signed symmetric quantization offers a third option: it maintains the runtime efficiency of symmetric quantization while strategically placing the extra representable value on the dominant-outlier tail through a lightweight sign selection rule, keeping the zero point at zero. Theoretical analysis demonstrates that this signed absmax grid is conditionally bound-optimal for L2 quantization error, a condition frequently met in LLM weight groups at low bit widths. Empirical validation across Qwen3, Qwen3.5, and Llama3 families shows improvements in perplexity and few-shot accuracy over standard unsigned symmetric quantizers, with no additional inference cost.