Reoptimization Algorithms Improve Contextual Bandits with Knapsack Constraints

Contextual Bandits with Knapsack problems involve online decision-making where a system must assign products to arriving customers, each with finite resource constraints. The challenge lies in simultaneously learning an unknown linear reward function (based on customer and product features) and making assignments to minimize revenue loss compared to an optimal policy that already knows the reward function. Previous work in related dynamic-pricing problems, which also utilize re-optimization, typically achieved an average regret of O(1/√T). This new research introduces a natural and straightforward extension of the Upper-Confidence-Bound (UCB) family of algorithms, specifically tailored for these problems. By leveraging re-optimization techniques, the proposed algorithm significantly improves upon existing bounds, achieving an average regret of O((ln T)^3 / T), where T is the horizon length. This substantial reduction in regret indicates a much faster convergence to optimal performance, making the algorithm more efficient and effective for real-world applications requiring joint learning and resource management.