New Fusion Method Prevents Modality Collapse in Multimodal AI.

Multimodal learning aims to improve AI model performance by combining information from different data types, such as text, audio, and video. However, a common challenge, termed "strong-modality collapse," occurs when the integration process degrades the performance of an otherwise dominant modality, sometimes leading to multimodal models underperforming even the best unimodal baseline. This research proposes a novel approach called Inverted Asymmetric Fusion (IAF) to address this issue. IAF avoids forcing mutual attention across modalities. Instead, it preserves the dominant modality by allowing it to pass through the fusion process unchanged, while weaker modalities are designed to attend to the dominant one as a contextual anchor. Additionally, IAF employs Modality-Aware Knowledge Distillation to strengthen the weaker modalities before fusion. Evaluations across various benchmarks with different modality hierarchies demonstrate that IAF effectively preserves the dominant modality's accuracy at its unimodal ceiling, unlike symmetric fusion methods which can degrade it significantly. IAF consistently improves performance over the strongest unimodal baselines, showcasing a robust solution for effective multimodal integration.