Learning Policy Representations in Imperfect-Information Games

This paper delves into the challenge of learning effective representations, or embeddings, for policies within the complex domain of two-player zero-sum games with imperfect information. Such games, like poker, involve hidden information and strategic decision-making. The authors make three key contributions to this area. Firstly, they present novel methods for generating datasets of policies tailored for specific games. Secondly, they propose techniques for learning these policy representations, essentially creating a compact, meaningful numerical summary of a policy's behavior. Finally, they introduce various downstream tasks designed to rigorously evaluate how effective and useful these learned representations truly are. The research evaluates these methods using classic poker variants, Kuhn and Leduc Poker. Despite employing relatively basic techniques, the findings indicate that the learned embeddings successfully capture useful behavioral characteristics of the policies. This work is noted as one of the first systematic comparisons of self-supervised learning approaches for policy representation in games, with code made publicly available for further extension.