Large Decision Model Achieves Scalable Multi-Task Reinforcement Learning

Recent advancements in large-scale sequence modeling have shown that a single model can effectively learn useful representations across highly diverse data distributions. Inspired by this progress, researchers are now exploring whether a unified transformer policy can be trained across vast collections of heterogeneous reinforcement learning (RL) environments, aiming for scalable multi-task learning. The study introduces LDM-v0, a "Large Decision Model" trained offline using trajectories collected from thousands of environments spanning multiple domains and modalities. LDM-v0 functions as a multi-task, multi-modal transformer policy, conditioned on historical observations, actions, rewards, and termination signals, and is trained through supervised next-action prediction on these offline trajectories. The researchers detail the infrastructure, automated data generation, model architecture, and training methodology. Crucially, LDM-v0, a single pretrained model, was shown to match the performance of independently trained task-specific reference policies across approximately 1,000 environments, including robotics, autonomous driving, inventory management, cybersecurity, trading, and video games. These results underscore the viability of large-scale offline pretraining for heterogeneous RL environments using a single transformer policy.