C$^3$PO Benchmark Reveals MLLM Cross-Modal Reasoning Flaws.

Researchers have introduced C$^3$PO, a new benchmark designed to rigorously evaluate the cross-modal reasoning capabilities of Multimodal Large Language Models (MLLMs). While MLLMs can process diverse sensory inputs like video, audio, images, and text, this benchmark reveals that their reasoning often remains heavily biased towards a dominant modality, leading to brittle performance in complex scenarios. C$^3$PO comprises 3,404 samples, specifically testing two critical abilities: information composition (fusing dispersed evidence from multiple modalities) and counterfactual conflict (resolving deliberate contradictions across modalities). Its unique paired structure and four-tier design enable targeted diagnosis of when and why cross-modal reasoning fails. The benchmark exposed a significant performance gap: humans achieved 88.64% accuracy, while the best MLLM (Gemini-3.1-Pro) reached only 73.17%, with open-source models performing even worse under conflict. Attention probes revealed that 86-95% of failures stemmed from modality dominance, where models committed to one modality and ignored contradictory evidence, concentrating most attention on text. This indicates that multimodal perception alone does not guarantee robust reasoning; architectures must foster sustained cross-modal attention to prevent premature decision collapse.