LaCache Accelerates Diffusion LLMs with Caching and Mixed Precision

A novel acceleration framework called LaCache has been introduced to enhance the efficiency of Diffusion Large Language Models (DLLMs), which are known for their parallel generation capabilities via Semi-Autoregressive (SAR) decoding. The core issue LaCache addresses is the substantial operator-level redundancy in existing DLLMs, where the entire sequence is recomputed during denoising steps, even when parts of the sequence remain unchanged. LaCache tackles this by implementing Lossless State Memoization (LSM), a caching mechanism that stores intermediate results such as embedding outputs, pre-attention states, and FlashAttention statistics. This allows the model to bypass redundant computations for invariant tokens without affecting output quality. Furthermore, LaCache integrates a per-group FP8 quantization strategy for FFN layers, optimizing for memory bandwidth. This combined approach results in a 1.3X speedup on its own, and when integrated with other acceleration methods, it can boost end-to-end performance by up to 40.2X, all while preserving task accuracy.