AI Misses Rare Cases in Chest X-rays, Highlighting Fairness Gaps

New research highlights a critical fairness issue in AI models used for chest X-ray (CXR) classification: even when these models show acceptable overall ranking performance, they often fail to diagnose rare conditions, particularly within certain patient subgroups. This problem, termed "thresholded subgroup underdiagnosis," occurs because rare-positive patients can fall below the diagnostic threshold. The study investigated this pre-deployment fairness problem as an audit question, analyzing who gets missed once a long-tailed multi-label CXR model converts scores into clinical decisions. Across two major CXR datasets, VinDr-CXR and MIMIC-CXR/CXR-LT, researchers employed a diagnostic ladder approach, combining class-level long-tail losses, subgroup-aware weighting, group robustness techniques, and tailored threshold selection. Results on VinDr-CXR showed significant improvements: group-tail weighting followed by tail-aware thresholding reduced the tail False Negative Rate (FNR) from 0.665 to 0.269, and worst-group FNR for sex and age also saw substantial reductions. While macro-mAP slightly increased, the study concluded that rare-label fairness in CXR is a complex issue dependent on the specific finding, patient subgroup, and the chosen operating threshold, rather than solely on label frequency or aggregate ranking metrics.