New Method Measures AI Explainer Stability and Reliability.

Attribution methods (AMs) are widely used to explain the decisions of black-box AI models by assigning importance scores to features. However, many AMs suffer from variability in their attribution scores due to stochastic components, making their explanations less reliable. This research introduces a novel distribution-based framework designed to quantify the stability of these attribution scores. The proposed method focuses on understanding the "separability" within ranked attribution vectors. It can identify the highest index at which a feature ranking remains dependable, providing a concrete measure of an explainer's reliability. Furthermore, the framework can be extended to compare different attribution methods based on how robust their rankings are across various datasets. Experiments demonstrate the practical application of this approach in evaluating the stability of AMs, offering a complementary criterion for assessing their trustworthiness.