Denoising Score Matching Loss Floor Linked to Fisher Geometry

New research delves into the fundamental limits of denoising score matching, a core training objective for diffusion models. The study identifies an "irreducible excess" in the training loss, which arises because the conditional score target remains random even at a fixed noisy state. This excess loss is shown to be directly equivalent to the trace of the Fisher-Rao metric of the conditional endpoint family, integrated along the diffusion trajectory. This discovery establishes a crucial link between the training objective's inherent limitations and the information geometry present within the latent spaces of diffusion models. By deriving this from a Schr"odinger bridge variational principle, the authors provide a conditional-variance decomposition of the denoising objective. For corruption diffusions, the Fisher term is proportional to the rate at which the noisy state loses mutual information about the clean data, offering insights into how information flow and corruption schedules influence the loss floor.