DLPMAC Clusters Incomplete, Disorderly Multi-View Data with Dual-Learning.

Real-world data often comes from multiple modalities (multi-view data), but challenges like missing modalities, inconsistent sampling frequencies, and network delays lead to incomplete and disorderly datasets. Traditional multi-modal fusion methods struggle with these issues, particularly in ensuring accurate sample-level alignment and handling significant data size discrepancies between different classes. Researchers propose DLPMAC (Dual-Learning based Penalized Multi-Align Clustering), a novel model that tackles these problems. DLPMAC incorporates a dual-learning mechanism, allowing it to learn semantic and structural prior knowledge from each modality. This mechanism helps preserve consistency and similarity across modalities at both local and global levels, which is crucial for effective fusion. Furthermore, the model features a penalized multi-align module that performs multi-to-multi data alignment. This module enables a single sample to form data pairs with various samples from other modalities, significantly improving alignment accuracy. The penalty mechanism within this module also prevents data aggregation, ensuring that excessive samples are not linked to a single sample. Experimental results confirm DLPMAC's effectiveness in addressing data alignment and fusion challenges from both sampling and clustering perspectives.