QuaSAR Improves Low-Bit Quantization Accuracy with Stable Compensation

Recent advancements in training-free post-training quantization (PTQ) aim to restore model accuracy by applying closed-form residual compensation. To manage the additional storage overhead, many existing methods selectively apply compensation only to layers where it yields a positive "fit score," discarding others. This paper reveals a critical flaw in this gating mechanism, particularly in low-bit settings like W4A4 (4-bit weights, 4-bit activations). It demonstrates that rank-deficient input activations can lead to severely ill-conditioned or numerically singular Gram matrices. This instability causes the closed-form solver to fail, producing spuriously negative fit scores. Consequently, layers that could benefit from compensation are incorrectly classified as uncompensable and discarded. To overcome this, QuaSAR introduces a parameter-free truncated pseudoinverse solver. This solver effectively removes collapsed directions before inversion, ensuring numerical stability. When applied to ViT-B with W4A4 quantization, QuaSAR achieves 81.42% top-1 accuracy, surpassing previous PTQ methods and even fine-tuning-based baselines. Furthermore, when combined with joint low-rank and quantization compression, it reaches a deployable operating point of 80.26% accuracy at a compact 54.7 MB, offering an excellent balance between model size and accuracy.