Temporal Difference Learning Enhances Diffusion Model Consistency and Sample Quality

Diffusion models typically rely on local denoising objectives, which can lead to inconsistencies across different time steps in the generation process. This lack of cross-time consistency can degrade the quality of generated samples, particularly when using fewer sampling steps. Researchers have introduced a novel temporal difference (TD) objective designed to address this issue. By reframing the diffusion process as a Markov reward process and denoising as a policy evaluation problem, they developed a unified TD approach applicable to both discrete and continuous-time diffusion models. Empirical results show that incorporating this TD training significantly enhances sample quality, as measured by FID scores. The benefits are particularly pronounced in scenarios with limited computation budgets, where fewer sampling steps are used, making it a practical improvement for various generative AI applications.