New Method Boosts MLLM Efficiency with Predictive Visual Token Pruning.

Multimodal Large Language Models (MLLMs) are powerful but computationally intensive, largely due to processing numerous visual tokens. Existing visual token pruning methods often rely on attention from a fixed middle layer of the language model, which is suboptimal because the most relevant layer varies by input. Furthermore, these methods still incur significant computational cost before pruning occurs. The new approach, Middle-layer Attention Prediction (MAP), addresses these issues. It first uses a "Question Contrastive Teacher Selection" mechanism to dynamically identify the best middle layer for attention for each specific input. Then, it distills the attention from this optimal layer into a lightweight predictor. This predictor estimates visual token importance from early multimodal features, allowing pruning to happen *before* the first language model layer, thus maximizing efficiency gains. MAP achieves a 3.09x speedup on LLaVA-NeXT-7B benchmarks while retaining 97.5% of the original model's performance.