Linear Attention Retrofitted into Diffusion LLMs for Faster Inference.

Diffusion Language Models (dLLMs) offer faster inference than autoregressive models through parallel decoding, typically using blockwise semi-autoregressive generation. However, even with KV caching, each denoising step still incurs a significant prefix-attention cost by attending to all previous blocks. This bottleneck limits further acceleration. To address this, a new approach introduces "block-hybrid attention," which combines exact softmax attention within the active denoising block with linear attention over previous blocks. This hybrid mechanism can be retrofitted into existing pretrained dLLMs with minimal post-training effort. For example, LLaDA-Hybrid, a modification of LLaDA 2.1, replaced six attention layers in just 60 hours. The retrofitted model maintains comparable benchmark performance across tasks like HumanEval, MBPP+, and CMATH. Crucially, a Triton implementation of LLaDA-Hybrid achieved up to 1.7 times higher decoding throughput and supported more concurrent requests before memory exhaustion. This demonstrates that linearizing attention in pretrained dLLMs is an efficient way to achieve faster inference and better memory utilization.