OmniFocus Compresses Omni-Modal LLM Tokens for Efficiency.

Omni-modal large language models (OmniLLMs), capable of processing both audio and video, face high inference costs due to the extensive token sequences generated from multi-modal inputs. Existing compression techniques often rely on unimodal guidance, which can overlook the temporal relevance of information and assume uniform information density across modalities. Researchers propose OmniFocus, a novel training-free method for query-guided token compression in OmniLLMs. OmniFocus independently assesses the importance of tokens for video and audio, allowing for a balanced compression design. This approach ensures that crucial modality-specific evidence is retained while maintaining audio-visual alignment, thereby addressing potential modality bias issues. Evaluations on the Qwen2.5-Omni model family across four audio-visual benchmarks demonstrate OmniFocus's effectiveness. It maintains robust performance even at low token retention ratios, outperforming current baselines on several key metrics. For instance, on DailyOmni with Qwen2.5-Omni-7B, OmniFocus achieved 59.40% accuracy at 25% token retention, while delivering up to 1.38 times faster prefill speed compared to the uncompressed baseline, showcasing a favorable balance between accuracy and efficiency.