New Protocol Diagnoses Misleading Offline Bandit Evaluations

Personalizing marketing messages using contextual multi-armed bandits (CMABs) offers significant business value, but evaluating these systems is complicated by delayed feedback, where the ultimate conversion metric might only be observed weeks later. This delay makes online learning difficult and can lead to misinterpretations from standard offline evaluation methods like off-policy estimates or confidence intervals. To address this, researchers propose an ordered diagnostic protocol. This protocol systematically screens reward and policy candidates along two critical dimensions: "alignment" (does optimizing the proxy reward truly advance the primary business objective?) and "learnability" (can the bandit effectively identify the optimal policy based on the chosen reward?). This process helps determine whether a CMAB is genuinely worth its complexity. The protocol was validated on a public benchmark and a synthetic generator, and illustrated on a large-marketplace push system. Two key lessons emerged: firstly, a denser reward signal, even if seemingly tied in static estimates, can significantly improve online learning. Secondly, what appears as personalization might sometimes be robustness, where a per-user policy primarily avoids betting on the wrong single message rather than truly personalizing, potentially overstating the "personalization premium."