New Benchmark Challenges Multimodal Time-Series Forecasting

Current benchmarks for multimodal time-series forecasting often suffer from issues like small-scale, synthetic data, limited types of textual context, and susceptibility to data leakage during evaluation. These limitations hinder the development of truly generalizable forecasting models. To address this, researchers have developed TimesX, a novel benchmark featuring a diverse collection of high-quality, real-world time series data. This data is enriched with varied textual contexts, generated through an automated pipeline, ensuring better representation of real-world scenarios and mitigating data leakage. An empirical study using TimesX revealed that many multimodal forecasting methods that perform well on older benchmarks struggle with this new, more rigorous evaluation. Surprisingly, simpler ensemble techniques that effectively utilize the rich textual context accompanying the time series data demonstrated superior performance on TimesX compared to more complex baselines.