Explorative Modeling Unlocks New Generative AI Scaling Axis.

While end-to-end training has revolutionized deep learning, generative modeling has largely remained an exception, often relying on factored generation procedures to handle multi-modal distributions. This approach can lead to "mode blurring" and prevents true end-to-end training. Researchers introduce Explorative Modeling (XMs), a novel paradigm that shifts the factorization from the generation procedure to the training loop itself. XMs explore multiple candidate matches between model outputs and data, then train on the most optimal match, allowing predictions to commit to specific modes rather than averaging them out. This approach establishes exploration as a "third pretraining axis," alongside parameters and data, for existing generative models. Scaling exploration consistently improves performance across diverse domains (images, video, language), with gains increasing significantly with model and data scale. XMs also enable end-to-end reconstructive generative modeling, matching diffusion model performance with substantially fewer inference steps, marking a significant advancement in generative AI.