BlockServe Boosts Diffusion LLM Serving Throughput

Serving diffusion large language models (dLLMs) efficiently is challenging due to varying convergence rates among batched requests. This "convergence heterogeneity" causes faster requests to wait for slower ones, leading to wasted compute resources and increased latency. Researchers have developed BlockServe, a novel continuous batching framework designed to overcome these limitations. BlockServe integrates block-grained scheduling, which allows for the immediate eviction of completed requests at block boundaries, preventing stragglers from holding up the entire batch. The system also features mixed-state execution, extending dual cache and parallel decoding to handle heterogeneous batches, and a compute-aware admission controller that optimizes batch capacity. Evaluations on various benchmarks show BlockServe achieving 1.9 to 10.6 times higher throughput compared to Fast-dLLM, while maintaining comparable generation quality, establishing block-grained scheduling as a key technique for high-throughput dLLM inference.