TS-Fault Benchmarks Time Series Forecasters Against Structural Faults

Time series forecasting (TSF) is critical for decision-making across various sectors, yet current model evaluations primarily rely on average error on clean data, assuming this predicts real-world reliability. This assumption often fails because actual faults are not random noise but structured events, involving temporal patterns, broken dependencies, regime changes, missing data, and causal propagation. This paper presents TS-Fault, a novel benchmark designed to rigorously evaluate TSF models against explicit, parameterized fault scenarios with controllable semantic difficulty. TS-Fault categorizes common failures into four modes along two axes: observation- vs. mechanism-level, and univariate vs. multivariate. These faults are strategically injected into prediction-critical windows using a unified importance score, allowing robustness to be tested against the specific structures models depend on. The evaluation of 21 models across 6 datasets, 4 fault modes, and 5 difficulty levels revealed three counter-intuitive findings. First, clean-data accuracy often anti-correlates with robustness. Second, clean rankings are preserved under observation-level faults but significantly reshuffled under mechanism-level faults. Third, all catastrophic failures occurred under mechanism-level faults, with foundation models, despite their high clean-data accuracy, exhibiting the greatest fragility. The code for TS-Fault is publicly available.