KARMA Explains Time Series AI Models by Capturing Temporal Dependencies

Many existing explainable AI (XAI) methods are not well-suited for time-series forecasting models because they implicitly assume feature independence, ignoring the inherent temporal dependencies and causal structures in sequential data. This oversight can lead to misleading or inaccurate explanations. Researchers have developed KARMA (K-Order Markov Approximations), a new XAI method specifically designed for time-series predictors. KARMA works by building a Markov surrogate model that accurately reflects the temporal relationships learned by the original predictor. This involves three key steps: identifying the minimal history length (K) sufficient for prediction, estimating the best-fitting K-order Markov transition kernel from the discretized history space, and deriving a five-level global explanation hierarchy from this kernel. Demonstrated with real-world weather data and complex synthetic datasets, KARMA effectively recovers the data's causal structure as learned by the model. It also outperforms established attribution methods like TimeSHAP in identifying temporal dependencies, providing more reliable and contextually appropriate explanations for time-series forecasting.