MMIR-TCM Enhances Traditional Chinese Medicine Diagnosis with AI

Traditional Chinese Medicine (TCM) diagnosis, especially through visual tongue inspection, suffers from subjectivity and a lack of reproducibility. Applying multimodal AI to TCM tasks like syndrome differentiation and prescription generation is further complicated by the semantic gap between visual features and textual reasoning, alongside a scarcity of large, standardized datasets. To overcome these hurdles, researchers have developed MMIR-TCM, a new framework designed to mimic the diagnostic workflow of TCM experts. MMIR-TCM employs a three-stage architecture. It begins with a training-free Memory-SAM module for robust tongue image extraction. This is followed by a fine-tuned Qwen3-VL model that generates structured tongue diagnoses. Finally, a Qwen3-based Retrieval-Augmented Generation (RAG) component provides evidence-grounded clinical decision support. The framework was developed and validated using MedTCM, a newly introduced large-scale multimodal dataset. To accurately assess clinical accuracy, a domain-specific metric, TDEU, was also created. Comprehensive experiments show MMIR-TCM significantly outperforms leading models like GPT-4o and Gemini 2.5 Flash, demonstrating its potential to enhance TCM clinical decision support.