Task-Aware LLM Quantization Improves Efficiency and Performance.

Deploying large language models (LLMs) often faces significant memory and computational constraints, making mixed-precision quantization (MPQ) a crucial technique. This research uncovers a phenomenon called the "Perplexity Illusion," where layers deemed important by perplexity-based sensitivity metrics show little correlation with those critical for complex reasoning tasks. Furthermore, an "Alignment-Diversity Tradeoff" is identified: using only task-specific calibration data can degrade performance, while incorporating general-domain data improves robustness. Based on these insights, the researchers propose TASA (Task-Aware Sensitivity Analysis), a two-level framework for optimizing LLM quantization. TASA first determines an optimal calibration-data mixture using a training-free gradient-trace alignment criterion. It then aggregates both perplexity and reasoning-oriented sensitivity signals to guide bit allocation at both inter-layer and intra-layer levels. Experiments with LLaMA-3-8B and Qwen2.5-7B demonstrate a "precision inversion," where appropriately allocated 3.5-bit models can achieve or exceed the performance of less task-aware 4-bit baselines, significantly improving accuracy on benchmarks like GSM8K.