FinVerse Benchmark Evaluates Financial Time-Series Models Realistically

The emergence of time-series foundation models has highlighted a critical need for benchmarks that can accurately assess their forecasting capabilities in real-world financial contexts. Current generic benchmarks often use uniform error-based metrics across diverse series, which may not reflect actual decision-making utility in finance. For instance, predicting stock price direction can be more valuable than merely minimizing point-wise error. To address this, FinVerse has been introduced as a finance-domain time-series forecasting benchmark. It comprises over 116,000 financial time series, with a subset of 60,000 economically relevant series designated for evaluation. Crucially, FinVerse defines 11 metric families, totaling 78 evaluation metrics, assigning the most appropriate ones to each series based on its economic meaning. Analysis of 43 public forecasting models using FinVerse demonstrated that models excelling in generic benchmarks do not necessarily produce effective financial forecasts, underscoring the importance of domain-aware evaluation.