RelShap Provides Relationally Consistent Shapley Explanations

This paper addresses a critical limitation in using Shapley values for explaining machine learning models, particularly when dealing with relational data. Traditional methods often flatten relational data into a single table, which discards crucial structural constraints. This can lead to Shapley value-based explanations that rely on feature independence, evaluating combinations of features that are impossible in the real-world data, thus producing misleading insights. The proposed framework, RelShap, overcomes this by incorporating relational constraints and data provenance directly into the Shapley value calculation. It ensures that both the background data and the feature coalitions evaluated are always relationally valid. RelShap is designed to be estimator-agnostic, meaning it can be composed with popular methods like Kernel SHAP, Monte Carlo, and Leverage SHAP without altering their core sampling or weighting properties. Furthermore, RelShap leverages functional dependencies within the data to create equivalence classes over feature coalitions, which significantly reduces runtime without compromising the accuracy of the Shapley values. Experiments across various datasets, models, and estimators demonstrate that RelShap generates explanations that are more faithful to the underlying data-generating process, correctly identifying dominant features where other methods, including Conditional SHAP and ManifoldShap, fail.