New Contextual Bandit Algorithm Improves Safety with Realized-Cost Constraints.

Traditional contextual bandit models often ensure safety by setting thresholds on expected costs. However, this approach can be insufficient in scenarios where the variability of outcomes, not just the average, is critical, such as in drug dosage or autonomous driving. This paper proposes a novel algorithm, High-Probability Constrained UCB, designed for contextual bandits with continuous actions. The new method focuses on enforcing high-probability constraints on the realized cost at each stage, rather than just the expected cost. This optimistic-pessimistic algorithm explores for rewards while conservatively identifying safe action sets. For linear models, it achieves a tight regret bound, and its applicability extends to general function classes. Experimental results demonstrate that this realized-cost safety approach substantially lowers the number of safety violations compared to existing baselines that only consider expected costs. This makes the algorithm particularly valuable for high-stakes applications where even a single unsafe decision can have severe consequences.