New Method Boosts Off-Policy Learning in Challenging Scenarios

This paper introduces a novel approach called Cross-Domain Off-Policy Evaluation and Learning (OPE/L) to address significant limitations in current contextual bandit systems. Traditional OPE/L methods struggle with scenarios like sparse data, policies that don't explore much, or when entirely new actions are introduced, often due to high variance or insufficient exploration in historical logs. The proposed framework overcomes these challenges by allowing the use of logged data not just from the target domain where a new policy will be deployed, but also from various other source domains. This is highly relevant in applications such as personalized medicine, content recommendation, and advertising, where data from different hospitals, countries, or user segments can be aggregated. By developing a new estimator and policy gradient method that integrates both target and source datasets, the researchers demonstrate substantially improved OPE/L performance. This cross-domain leverage allows for more robust and effective evaluation and optimization of new policies, even in data-scarce or under-explored environments.