LC-GRPO Improves Flow-Based Generative Models in Reinforcement Learning.

Flow-based generative models typically use deterministic ODEs for sampling, but online reinforcement learning (RL) requires stochastic rollouts for exploration. Existing GRPO methods for flow models replace the ODE with an SDE during training, leading to a significant mismatch between blurry SDE training samples and sharp ODE inference samples. LC-GRPO addresses this train-inference gap by introducing a Langevin correction. Each rollout transition first takes an inference-aligned ODE Euler step, then applies a stochastic Langevin correction targeting the marginal distribution at that timestep. This correction recovers the necessary score directly from the flow velocity without needing an additional score model. The proposed method theoretically reduces Wasserstein error and offers more accurate transitions than standard Euler-Maruyama discretization. Empirical results on text-to-image and text-to-video tasks (SD3.5-Medium, FLUX.1-Dev, HunyuanVideo) show LC-GRPO consistently improving reward optimization, maintaining generation quality, and significantly narrowing the gap between training and inference.