New Metric Assesses AI Explanation Consistency for Medical Imaging Fairness

Evaluating fairness in medical imaging AI typically focuses on how well models perform across different demographic subgroups. However, this approach doesn't reveal whether the AI relies on the same visual cues or evidence when making predictions for various groups. This paper addresses that gap by proposing a new metric called the Explanation Consistency Score (ECS). The ECS uses Jensen-Shannon divergence to measure the similarity of attribution maps, which highlight the parts of an image an AI focuses on, across different demographic subgroups. The researchers applied this metric to a case study involving diabetic retinopathy screening. Their experiments revealed that while the AI's predictive performance did show differences across ethnic groups, the explanation consistency remained relatively high and did not strongly correlate with these performance disparities. This suggests that predictive fairness and explanation consistency are distinct but equally important dimensions of AI behavior, advocating for a broader approach to fairness evaluations beyond just predictive accuracy.