Experimental Analysis Compares Diffusion Language Models Performance

While autoregressive Large Language Models (LLMs) have dominated language generation, Diffusion Language Models (DLMs) offer an alternative paradigm, generating text through iterative denoising. However, comparing the capabilities and trade-offs of various DLM architectures has been challenging due to inconsistent evaluation protocols, datasets, and inference settings across different studies. This research provides a systematic experimental analysis to address this gap. It evaluates eight leading DLMs across eight diverse benchmarks, covering tasks such as reasoning, coding, translation, and structured problem-solving. The analysis explicitly considers both the quality of the generated output and the computational efficiency of each model. Beyond benchmark performance, the study delves into the impact of crucial inference-time factors, including the number of denoising steps, context length, block size, and parallel unmasking strategies. By conducting large-scale experiments and controlled comparisons of smaller models, the research highlights the distinct strengths and limitations of diffusion-based language modeling across different tasks and inference budgets, offering practical insights for their deployment.