New Bandit Algorithm Handles Drifting Rewards and Actions

Many real-world online decision-making scenarios involve constantly changing conditions, such as evolving user preferences or fluctuating available actions. Traditional bandit algorithms often struggle when both the feasible actions and the underlying reward models drift over time. Existing solutions that achieve optimal performance typically rely on restrictive assumptions about the structure of decision sets. This paper presents a novel method that overcomes these limitations for non-stationary linear bandits. It introduces a unified "misspecification-reduction" perspective, where the problem horizon is divided into blocks. Within each block, the dynamic regret is related to a fixed-parameter linear bandit benchmark, with parameter drift treated as bounded misspecification. This approach allows for optimal dynamic regret guarantees for a broader range of applications, including those with general compact decision sets and K-armed contextual linear bandits, making it more applicable to complex, dynamic environments.