C$^2$MOE Enhances Multimodal Emotion Recognition with Incomplete Data

Multimodal Emotion Recognition in Conversations (MERC) systems typically rely on complete multimodal inputs, but real-world data often suffer from missing modalities due to transmission errors or user behavior. This incompleteness severely degrades model performance, and existing methods, while enhancing robustness through cross-modal consistency learning, frequently overlook modality complementarity, leading to biased data reconstructions. To overcome these limitations, a new framework called C$^2$MOE (Consistency and Complementarity-guided Mixture of Experts) has been introduced. This novel approach unifies representation learning and missing modality imputation within a principled information-theoretic framework. It factorizes multimodal knowledge into distinct consistency and complementarity components using interaction-aware experts. Consistency is achieved by maximizing cross-modal predictability, while complementarity is preserved by maximizing conditional entropy between modalities. C$^2$MOE features a dual-branch prediction mechanism for robust imputation: a consistency branch aligns imputed features with the joint distribution by minimizing uncertainty, and a complementarity branch exploits unique modality cues via entropy maximization. Finally, a learnable reweighting module dynamically assigns importance scores to each expert's output, resulting in a robust and adaptive fusion for imputation. Extensive experiments on multiple MERC benchmarks confirm that C$^2$MOE consistently surpasses state-of-the-art methods across various missing-modality scenarios, validating its robustness and generalization capabilities.