GenDa Improves Unsupervised RL for Generalizable, Data-Efficient Skill Learning

Unsupervised Reinforcement Learning (URL) aims to pre-train versatile, skill-conditioned policies without relying on explicit rewards, serving as a foundational step for various downstream control tasks. Despite recent advancements, current off-policy URL methods face two critical, often overlooked, limitations: the non-stationary nature of skill semantics and a lack of robust generalization. These bottlenecks hinder the scalability and practical applicability of URL. To tackle these challenges, researchers have developed GenDa (Generalizable Data-efficient Agent), a unified framework designed for robust unsupervised reinforcement learning. GenDa introduces a novel skill relabeling mechanism that effectively mitigates non-stationarity, leading to significantly improved data efficiency during the pre-training phase. Furthermore, it incorporates a Complementary Information Bottleneck (CIB) which encourages the learned skill policy to focus on ego-centric features, thereby enhancing its robustness to distribution shifts encountered in subsequent tasks. Experiments demonstrate GenDa's superior generalizability and data efficiency, significantly boosting URL scalability.