Few-Step Text Latents Fail Due to Sharp Categorical Readouts

This paper investigates a fundamental difference in how few-step generative models perform with text versus images. While image generation from continuous latents can be deterministic and efficient, similar approaches for text often result in incoherent output. The researchers pinpoint the problem not as a training or scaling issue, but as a geometric one: text decoders face a challenge in making discrete categorical choices when the underlying latent space is continuous and requires a sharp readout. They introduce new metrics, DABI (readout sharpness) and CCI (categorical commitment), to quantify this phenomenon. Their findings indicate that text decoders significantly amplify small perturbations near decision boundaries, unlike image decoders which maintain smoother transitions. This amplification prevents stable few-step deterministic generation for text. The study explores two primary mechanisms to overcome this limitation: categorical commitment, as seen in autoregressive decoders, and stochastic re-injection, which introduces randomness to improve performance. They also provide theoretical proofs on the accuracy-depth-stiffness tradeoff inherent in deterministic-continuous models, suggesting that current deterministic methods are fundamentally limited for text generation without stepping outside this class.