New Method Improves RL Pre-training with Temporal Correlation Learning.

Unsupervised pre-training on vast video datasets is a promising avenue for enhancing the sample efficiency and overall performance of Reinforcement Learning (RL) models. However, current pre-training techniques, which often rely on single-step transition prediction or image reconstruction, tend to emphasize large, static information within the pixel space. This can lead to the neglect of smaller, yet critically important, dynamic details. To address this, researchers introduce a novel approach that shifts focus from raw pixels to temporal correlations. They propose the Multi-scale Temporal Contrastive Learning (MTCL) method, designed to model these correlations across various scales. By doing so, MTCL ensures that all elements within a video receive balanced attention, resulting in more informative representations. This improved representation learning has been shown to effectively support policy learning, leading to better sample efficiency and asymptotic performance across a range of downstream RL tasks.