Foundation Models Struggle with Extreme Wildfire PM2.5 Forecasting

A new study has systematically evaluated the generalizability of time series foundation models (TSFMs) in forecasting extreme environmental events, specifically focusing on PM2.5 concentrations during California wildfires. Wildfire smoke events lead to hazardous PM2.5 levels, which are notoriously difficult to predict due to their rare and extreme nature. The research benchmarked six TSFM configurations, including zero-shot and LoRA fine-tuned versions of TimesFM, Chronos-2, Moirai-2, and Time-MoE, against traditional baselines like LSTM, BiLSTM, Transformer, and naive persistence. The evaluation used a 12-year hourly PM2.5 dataset covering 1,375 wildfire incidents across 79 California monitoring sites. A leave-one-incident-out protocol was employed to assess generalization to unseen fires, using metrics such as MAE, RMSE, and exceedance F1 scores at EPA AQI thresholds for 6-, 12-, and 24-hour horizons. The findings revealed a consistent hierarchy: the BiLSTM baseline achieved the lowest MAE and highest exceedance F1 across all thresholds, including the hazardous band, significantly outperforming all foundation models. Zero-shot TSFMs showed only marginal improvements over simple persistence and, in the case of zero-shot Chronos-2, exhibited severe RMSE tail instability due to sporadic large errors. While LoRA fine-tuning improved the adapted TSFMs and mitigated some instability, no foundation model managed to surpass the performance of the trained recurrent baselines on any metric. This challenges the assumption that large pretrained models universally excel in environmental forecasting, especially under extreme, out-of-distribution conditions.