Ternarization Reduces Qwen3-4B Size, But Performance and Speed Vary

Anirudh Malik, M Sparsh Mehra, Poojith Devan· September 3, 2026 View original

Key takeaways

  • Post-training ternarization can significantly reduce LLM storage size.
  • This compression often leads to uneven degradation in task accuracy and increased perplexity.
  • Reduced model size does not automatically mean faster inference; runtime performance must be separately benchmarked.
  • Careful trade-off analysis between model size, accuracy, and speed is essential for deployment.

Who benefits

Edge AIMobile ComputingCloud ComputingAI/ML EngineeringIoT

Summary

This study explores post-training ternarization of the Qwen3-4B language model, achieving significant storage reduction (from 8.29 GiB to 3.96 GiB) but observing uneven degradation in task capability and increased perplexity. Preliminary benchmarks also indicate slower inference speeds despite compression.

The research investigates the post-training ternarization of Qwen3-4B, a 4-billion parameter instruction-tuned language model, aiming to reduce its storage footprint and memory bandwidth requirements. The process involved techniques like KOTMS rotation, E2M-ATQ ternarization, and GPTQ-style error compensation, focusing on weight-only quantization while keeping activations at 16-bit precision. The study successfully reduced the model size from 8.29 GiB to 3.96 GiB, achieving an effective bit budget of 1.641 bits per weight for 81.62% of parameters. However, this compression came with performance trade-offs. Across ten capability comparisons, accuracy dropped from 64.5% to 54.7%, with some tasks like BoolQ retaining 84.6% of performance while ARC-Challenge only retained 43.8%. Perplexity also increased across various datasets. Crucially, while storage was significantly reduced, preliminary microbenchmarks showed a 4.6x slower inference speed compared to FP16 cuBLAS on one tested shape. This indicates that compression alone does not guarantee faster inference, and further optimization for runtime behavior is needed. The packed artifact's end-to-end task accuracy and generation throughput were not fully benchmarked.

Why it matters

For professionals deploying large language models in resource-constrained environments, understanding the trade-offs of ultra-low-bit quantization is crucial. This research highlights that while storage can be significantly reduced, careful evaluation of performance degradation and potential inference slowdowns is necessary.

How to implement this in your domain

  1. 1Evaluate the specific task performance requirements before applying aggressive post-training quantization techniques like ternarization.
  2. 2Benchmark both storage reduction and inference speed post-quantization, as compression does not automatically translate to faster runtime.
  3. 3Prioritize tasks where a slight drop in accuracy is acceptable in exchange for significant memory savings.
  4. 4Explore activation quantization and hardware-specific optimizations in conjunction with weight quantization to achieve end-to-end performance gains.

Original post by Anirudh Malik, M Sparsh Mehra, Poojith Devan

"arXiv:2609.01962v1 Announce Type: new Abstract: Ultra-low-bit language models can reduce storage and memory bandwidth, but a nominal "1.58-bit" label does not fully describe the stored representation, retained capability, or runtime behavior. We study an end-to-end post-training…"

View on X

Originally posted by Anirudh Malik, M Sparsh Mehra, Poojith Devan on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses