Latent-Kernel Discrete Flow Maps Boost Few-Step Text Generation

Traditional discrete diffusion and flow-matching models, used for denoising sequences, often struggle with few-step generation in text. This is because they factorize transitions and decide tokens independently, leading to mismatches when positions are coupled, such as subject-verb agreement. Existing few-step methods typically rely on distilling or rectifying slower, high-quality teachers, inheriting their performance ceiling. This research introduces Latent-Kernel Discrete Flow Maps (LKF), a novel flow-map kernel designed from scratch to natively handle correlated steps. LKF uses a mixture of factorized components, all tied by a single shared latent variable. When conditioned on this latent, each component is computationally inexpensive, and the mixture can be summed in closed form for a small number of components. The LKF model demonstrates that a single step can effectively place mass on correlated completions with the same sampling time complexity as a factorized model. Experiments in unconditional text generation on benchmarks like One-Billion-Word and WikiText-103 show that LKF learns highly heterogeneous components, improving generative perplexity by 2.1x to 3.3x over likelihood baselines without sacrificing diversity. The performance gains increase with the number of components, with LKF at M=8 surpassing distilled and rectified few-step samplers.