MGDT Enhances Multimodal Knowledge Graph Completion with Diffusion Transformers

Xu Hou, Meiyu Liang, Wei Huang, Yawen Li, Zhe Xue, Wu Liu, Guanhua Ye, Lei Shi, Kangkang Lu· July 20, 2026 View original

Summary

MGDT is a novel framework for Multimodal Knowledge Graph Completion (MKGC) that uses a Multimodal Large Language Model (MLLM)-guided diffusion transformer with a Relation-Adaptive Mixture-of-Experts. It improves performance by aligning multimodal representations and performing graph-conditioned denoising, addressing limitations of existing diffusion-based methods.

Multimodal Knowledge Graph Completion (MKGC) involves inferring missing entities by integrating structural, textual, and visual information. Traditional diffusion-based MKGC methods often struggle because their denoisers must simultaneously handle relation-dependent cue selection, cross-modal semantic alignment, and structure-aware entity generation, leading to noisy and inconsistent conditions. To overcome these challenges, researchers propose MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts. MGDT employs an "align-then-diffuse" paradigm. First, a Relation-Adaptive Semantic Routing Mixture-of-Experts (RASR-MoE) module intelligently selects relevant multimodal semantic transformation paths, effectively suppressing irrelevant modality interference. Second, a frozen Multimodal Large Language Model (MLLM) acts as a semantic anchor, aligning these routed multimodal representations into a unified latent space, thereby reducing cross-modal semantic heterogeneity. Finally, a Knowledge Graph Diffusion Transformer (KGDT) performs graph-conditioned denoising generation within this aligned space to produce the representation of the missing entity. Experimental results across three benchmark datasets consistently show that MGDT significantly outperforms existing strong baselines, demonstrating its effectiveness in MKGC.

Why it matters

This advancement provides a more robust and accurate method for completing knowledge graphs by effectively integrating diverse data types, which is crucial for sophisticated AI applications requiring comprehensive understanding.

How to implement this in your domain

  1. 1Investigate integrating advanced multimodal knowledge graph completion techniques into data management and AI development pipelines.
  2. 2Explore the use of Mixture-of-Experts architectures for handling diverse data modalities in complex AI systems.
  3. 3Consider leveraging frozen MLLMs as semantic anchors to align heterogeneous data representations for improved model performance.
  4. 4Apply graph-conditioned diffusion models for generating missing information in structured data environments.
  5. 5Benchmark existing knowledge graph completion solutions against MGDT's approach for potential performance gains.

Who benefits

Data ManagementSemantic WebE-commerceHealthcareFinancial Services

Key takeaways

  • MGDT improves multimodal knowledge graph completion by addressing noise and semantic inconsistency.
  • It uses a Relation-Adaptive Mixture-of-Experts for intelligent cue selection.
  • A frozen MLLM aligns diverse multimodal representations into a unified space.
  • Graph-conditioned diffusion transformers generate missing entity representations.

Original post by Xu Hou, Meiyu Liang, Wei Huang, Yawen Li, Zhe Xue, Wu Liu, Guanhua Ye, Lei Shi, Kangkang Lu

"arXiv:2607.15592v1 Announce Type: new Abstract: Multimodal Knowledge Graph Completion (MKGC) requires inferring missing entities from structural, textual, and visual cues. Existing diffusion-based MKGC methods usually denoise directly on raw multimodal features. Such a design for…"

View on X

Originally posted by Xu Hou, Meiyu Liang, Wei Huang, Yawen Li, Zhe Xue, Wu Liu, Guanhua Ye, Lei Shi, Kangkang Lu on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses