Activation Outliers Key to Robust Quantization in MLLMs.

The increasing demand for efficient Multimodal Large Language Models (MLLMs) has driven interest in low-bit quantization to reduce computational and memory footprints. While hardware support for formats like MXFP8 and MXFP4 is growing, aggressive 4-bit quantization often leads to significant performance drops. This research systematically investigates these quantization schemes, revealing that activation quantization, rather than weight quantization, is the dominant factor causing performance degradation. Motivated by this insight, the authors introduce Residual Fallback Quantization (RFQ), a novel and lightweight framework for activation reconstruction. RFQ augments the primary ultra-low-bit activation representation with an auxiliary quantized residual pathway, effectively modeling and compensating for quantization errors. This approach significantly improves activation fidelity while maintaining the efficiency benefits of low-bit computation. RFQ requires no architectural modifications and incurs negligible overhead, demonstrating substantial performance recovery in MLLMs like Wan2.2 and Qwen3-VL across generation and reasoning tasks, thereby narrowing the gap to higher-precision baselines.