ExTernD Achieves Near BF16 Accuracy for Ternary LLM Quantization

Large Language Models (LLMs) often require significant computational resources and memory, prompting research into quantization techniques to reduce their footprint. One such approach is ternary quantization, which constrains weights to -1, 0, or +1. However, fixed-bit ternary schemes typically have inherent accuracy limitations. This new research introduces ExTernD, or Expanded-rank Ternary Decomposition, a post-training quantization method designed to overcome these limitations. ExTernD factorizes each LLM weight matrix into three components: two ternary matrices (B and C) and a real-valued diagonal scaling vector (D). Crucially, the inner rank (k) of this decomposition is deliberately expanded beyond the full rank of the original matrix. This expansion allows subsequent components to correct the quantization errors introduced by earlier ones, enabling a continuous reduction in the residual error. The theoretical foundation of ExTernD demonstrates that this residual error decreases monotonically with increasing k, meaning the method can achieve accuracy arbitrarily close to bf16 precision. The method offers continuous control over memory and compute scaling through the expansion factor (mu) and sparsity through a threshold (tau), allowing for precise targeting of accuracy levels rather than being limited to discrete bit-widths. Empirical evaluations on models like Gemma-4-E2B and Qwen3.5-4B show that ExTernD matches the per-matrix accuracy of Q4_K at an effective bit-width of 5.2-5.5 bits per weight. A full Qwen3.5-4B conversion with mu=3 achieved a Wikitext-2 perplexity of 10.10, very close to the bf16 baseline of 9.78, positioning it within the accuracy band of Q4_K/Q5_K at approximately 5.7 effective bits per weight.