New Method Boosts MLLM Efficiency by Optimizing KV Cache Selection

Multimodal Large Language Models (MLLMs) face significant memory bottlenecks due to the large number of visual tokens in their Key-Value (KV) cache, which grows linearly with context length. Existing KV selection methods often estimate importance during the prefill stage, assuming this accurately reflects decoding-time queries. However, this research highlights that decoding-time queries in MLLMs exhibit much greater variance, leading to unstable importance estimation and the potential discarding of crucial visual tokens, which degrades performance. To counter this, MM-ShiftKV is introduced as a training-free, decode-aware KV selection method that operates strictly during prefilling. It constructs "variance-expanded query proxies" to better anticipate decoding-time query behavior and estimates prompt KV importance based on their aggregated attention mass. Experiments show that MM-ShiftKV consistently outperforms other methods under tight KV-cache budgets, preserving grounding and reasoning performance by preventing the premature discarding of semantically vital visual information.