New Algorithm Optimizes Embedding Model Routing in Recommenders

Modern recommendation systems frequently route diverse user queries to multiple embedding models to provide relevant suggestions. This paper addresses the challenge of optimizing this dynamic routing under realistic conditions, such as adversarial queries, bandit feedback, and limited model observability. The authors formalize this problem as an adversarial contextual linear bandit, where queries are contexts, items are actions, and embedding models act as low-rank experts. The research identifies that standard regret notions are insufficient for this problem and proposes a log-quadratic policy class that is expressive yet structured enough for efficient online learning. To solve this, they introduce Hypentropy Policy Gradient (HPG), a policy gradient algorithm. HPG is proven to adapt to unknown low-rank structures even with incomplete information, achieving efficient linearized policy regret that avoids the curse of dimensionality. The paper also provides a computationally efficient and parameter-free implementation of HPG, making it a practical solution for dynamic embedding model routing.