FedTCR Advances Federated Multimodal Graph Learning

Federated Multimodal Graph Learning (FMGL) is an emerging field that extends federated learning to multimodal-attributed graphs (MAGs), allowing collaborative model training across decentralized datasets without sharing raw data. However, applying existing federated graph learning methods directly to FMGL is insufficient due to the inherent complexities of decentralized MAGs, which exhibit significant heterogeneity in tasks, data modalities, and graph topologies. To address these challenges, researchers propose FedTCR (Federated multimodal graph learning with Topology-aware Cross-modal Routing), a novel algorithm specifically designed for FMGL. FedTCR employs a two-stage approach: an initial federated task-agnostic pre-training phase followed by isolated task-oriented fine-tuning. Crucially, it introduces a topology-aware cross-modal routing mechanism to manage modality and topology heterogeneity. This mechanism allows each client to distill modality-specific knowledge into prototypes, which the server then uses to evaluate cross-client, cross-modal relationships and route informative prototypes for a tri-level contrastive learning scheme. This scheme aligns cross-client modalities while preserving discrimination, leading to superior performance across various graph-centric and modality-centric tasks.