Weight of Evidence Improves Feature Importance Explanations.

Explainable AI (XAI) relies heavily on Feature Importance Methods (FIMs) to interpret model predictions, but simply looking at attribution scores often doesn't provide deep insight into a model's reasoning. This research proposes a new perspective by integrating FIMs into a hypothesis-testing framework using the concept of Weight of Evidence (WoE). The WoE framework quantifies how strongly observed evidence supports a hypothesis about feature importance, whether that hypothesis comes from domain knowledge, ground truth, or the FIM itself. This allows for a more principled evaluation of FIMs, capturing both how well they align with prior expectations and how stable their explanations are. The paper also provides theoretical results connecting WoE to attribution variance, offering a deeper understanding of explanation reliability. Empirical tests with LIME and SHAP demonstrate the flexibility and applicability of this evidence-based, contrastive approach, providing a valuable complementary tool for XAI practitioners.