DiLaServe Boosts Performance for Diffusion Language Models

A new cluster-level serving system, DiLaServe, has been developed to optimize the deployment and performance of Diffusion Language Models (DLMs). DLMs are emerging as a promising alternative to traditional autoregressive models due to their ability to generate multiple tokens in parallel, offering higher inference throughput. DiLaServe is specifically designed to overcome the unique challenges associated with DLMs. These include navigating the inherent speed-quality tradeoff in confidence-based denoising, dynamically adjusting parallelization levels across model instances to manage fluctuating loads, and efficiently coordinating approximate KV caching mechanisms that introduce non-uniform costs per step. Through features like deadline-aware scheduling, adaptive load control via confidence-threshold adjustment, and dynamic cluster reconfiguration, DiLaServe achieves substantial improvements. Benchmarks show it can boost Service Level Objective (SLO) attainment by up to 56.6 percentage points and reduce end-to-end request latency by up to 46%, all while maintaining high accuracy.