UpliftBench Reveals Mismatches in Uplift Model Evaluation Metrics

Uplift modeling, which estimates conditional-average-treatment-effects for personalized targeting, is crucial for many business applications. However, existing benchmarks often show conflicting results regarding the best-performing estimators. UpliftBench, a new evaluation framework, demonstrates that these discrepancies largely stem from the choice of evaluation metrics rather than fundamental differences in model capabilities. The research, using an outer-test-isolated, multi-objective protocol across diverse datasets, found that metrics like Qini show poor alignment with effect accuracy on standard continuous benchmarks. While AUUC performs better, ranking metrics generally prove insufficient for sign-threshold policies because they disregard score levels. Calibrating decision thresholds significantly improves policy selection. UpliftBench provides versioned loaders, fixed protocols, and a reproducible leaderboard to standardize future evaluations.