FlexLAM Improves Latent Action Learning with Variable-Length Codes

This research introduces FlexLAM, a novel approach designed to overcome a fundamental limitation in existing Latent Action Models (LAMs). Traditional LAMs often face a "bottleneck trade-off" where a fixed-capacity latent action representation must balance between being too restrictive, potentially losing crucial transition information, or too expansive, which can complicate action alignment, especially with limited labels. FlexLAM addresses this by replacing the fixed-capacity bottleneck with variable-length latent actions. The core innovation of FlexLAM lies in its training methodology, which employs nested dropout to generate prefix-valid codes. This allows the model to first capture the essential, compact structure of transitions and then progressively incorporate finer details only when the complexity of the task demands it. This adaptive capacity is achieved without the need for new architectural designs or specialized loss functions, making it a straightforward enhancement. Experimental results demonstrate that a single FlexLAM model can match or even surpass the performance of multiple separately trained fixed-capacity LAMs across various token budgets. This holds true under standard scarce-label supervision and even in challenging low-return, single-task alignment scenarios. Furthermore, FlexLAM supports dynamic adjustment of the token budget during inference without requiring retraining, and it shows improved transition reconstruction on the Ego4D dataset. These findings suggest that variable-length latent actions, as implemented by FlexLAM, offer a simple yet powerful upgrade to the fixed-capacity bottleneck in various latent action and video-pretrained models.