Fidelity Metrics Fail to Predict Quantized LLM Performance in Critical Zone

In the deployment of quantized Large Language Models (LLMs), fidelity metrics such as per-token KL divergence (KLD) are frequently used as a low-cost proxy for assessing benchmark quality. This practice was tested across various quantized versions of Qwen3.6-35B-A3B and Devstral-Small-2-24B models, evaluated against a suite of downstream benchmarks. The findings indicate that KLD shows a strong correlation with benchmark scores when considering the full range of quantization levels. However, this correlation significantly diminishes and becomes statistically non-significant within the "silent zone," which refers to the performance range near the high-precision baseline. This breakdown in correlation was observed across 14 different measurement variations, including various KLD aggregations, perplexity formulations, and calibration corpora. Further analysis at the per-prompt level showed that KLD has only weak predictive power for identifying failures in code generation tasks and is ineffective as a cross-model router. The study attributes this collapse to KLD primarily measuring the volume of disagreement with a reference model, rather than the direction of those disagreements, especially in the critical silent zone where subtle performance shifts matter most.