Parallel-In-Time Sampling Accelerates Discrete Diffusion Models.

Discrete diffusion models are powerful tools for learning and generating discrete distributions, but their inherently sequential generation process often limits their speed. Researchers have developed a novel method to accelerate these models by introducing parallel-in-time sampling. This approach specifically targets the mainstream τ-leaping algorithm, which is used for absorbing discrete diffusion within a Continuous-Time Markov Chain (CTMC) framework. The acceleration is achieved by leveraging the continuous-time stochastic integral form of the τ-leaping algorithm and applying the Picard iteration method. This allows for parallel computation across time steps, significantly improving the overall time complexity. Empirically, the new sampler demonstrates substantial speedups, achieving 7-9x faster runtime for synthetic distributions. For real-world image and text generation tasks, it maintains comparable quality while offering 1.45-1.86x runtime speedups and requiring 50% fewer Negative Function Evaluations (NFEs) on a single GPU. This advancement has broad implications for efficient parallel inference in areas like molecular structure and language generation.