Holtercare-Bench: New Multimodal Benchmark for Long-Term ECG Analysis

Multimodal large language models (MLLMs) have shown promise in medical applications, but they often struggle with dynamic, long-term signals like electrocardiograms (ECGs). This limitation stems from a scarcity of high-quality datasets and benchmarks specifically designed for complex temporal reasoning and diagnostic report generation in this domain. To address this critical gap, researchers have developed two key resources. First, they introduce Holtercare-23K, a large-scale multimodal dynamic ECG dataset. This dataset comprises 22,980 question-answer pairs derived from 788 clinical Holter records, uniquely featuring a novel signal-video-text tri-modal alignment. This rich data provides a comprehensive foundation for training and evaluating MLLMs. Second, based on this dataset, they present Holtercare-Bench, a multimodal benchmark designed to rigorously evaluate models across three crucial tasks: temporal localization of events, clinical diagnosis, and global summarization of ECG findings. Initial zero-shot evaluations of leading MLLMs on Holtercare-Bench revealed significant performance deficiencies when processing ultra-long pathological sequences. However, subsequent fine-tuning of representative models demonstrated substantial improvements, underscoring the potential for specialized training. This work not only illuminates the current limitations of MLLMs in electrophysiology but also establishes a foundational benchmark for advancing long-term medical MLLMs.