StraightDP Enhances Private Generative Model Utility

Differentially private (DP) training of text-conditioned generative models often faces a significant drop in utility when strong privacy guarantees are required. This research re-examines the problem through the lens of rectified flows, observing that the Bayes-optimal velocity along the interpolation between noise and data exhibits heterogeneous information. At the noise end, a few class-conditional moments dominate, while closer to the data end, sample-specific structures become more critical.StraightDP leverages this heterogeneity by employing a two-pronged approach. A small portion of the privacy budget is used to release whitened class-conditional moments once, which can then be distilled into model weights or injected during sampling. The remaining, larger portion of the budget is allocated to pre-declared DP-SGD for training the model closer to the data end, where sample-specific details are crucial and beyond the reach of simple moments.Experiments on MNIST at an epsilon of 1 demonstrate the effectiveness of StraightDP. The released moments alone achieved 0.76 downstream accuracy and an FID of 237, significantly outperforming uniform DP-SGD which only reached 0.21 accuracy. The full StraightDP pipeline achieved 0.81 accuracy at FID 56 in a public latent space. Furthermore, applying per-token stream norm constraints to the multimodal backbone improved downstream accuracy in extreme-noise pixel-space regimes, with benefits increasing as privacy strengthened. The released moments also proved portable to frozen SD3-medium, where sampling-time injection surpassed DP-LoRA training with a fraction of the budget.