Top-k Pareto Bandits Optimize Multi-Objective Slate Selection.

This research addresses a complex decision-making challenge in dynamic environments: the stochastic multi-objective bandit problem, specifically tailored for selecting a "slate" of $k$ actions (arms) simultaneously. Unlike traditional bandit problems that aim for a single optimal arm, this work focuses on identifying and maintaining a small subset of actions that collectively approximate the Pareto frontier, which represents the set of optimal trade-offs across multiple objectives. The effectiveness of this subset is quantified using the dominated hypervolume, a metric that captures the volume of the objective space dominated by the selected arms. To tackle this, the authors introduce an algorithm called THV-UCB (Top-k Hypervolume Upper Confidence Bound). This optimistic algorithm greedily selects arms based on their optimistic estimates of marginal hypervolume contributions. The paper provides strong theoretical guarantees for THV-UCB, establishing a gap-free regret bound of $\tilde{O}(d\sqrt{nkT})$ that holds universally, where $d$ is the number of objectives, $n$ is the total number of arms, $k$ is the slate size, and $T$ is the number of rounds. Additionally, a gap-dependent bound of $\tilde{O}(nk^{2.5}/\Delta_{\min})$ is derived, which becomes polylogarithmic in $T$ when arms are sufficiently distinct. These theoretical results provide robust support for using small subsets of actions to effectively approximate Pareto fronts, offering significant implications for various multi-objective applications where balancing competing goals is crucial.