FourTune Enables Efficient 4-Bit Post-Training for Diffusion Models

Diffusion models have become a leading technology for generating high-quality content, but adapting these large models for specific applications through post-training remains computationally intensive. Existing parameter-efficient fine-tuning methods only partially alleviate the prohibitive memory footprints and slow training speeds. To tackle these challenges, a new framework called FourTune has been developed, enabling fully 4-bit efficient post-training for diffusion models. FourTune employs an end-to-end W4A4G4 paradigm, meaning weights, activations, and gradients are all processed in 4-bit precision. Its core innovation is a triple-branch hybrid pipeline that augments the standard LoRA architecture with a frozen numerical stabilizer. This stabilizer isolates quantization-sensitive outliers, ensuring stable training even under native 4-bit computation. Furthermore, FourTune incorporates hardware-efficient block-wise quantization and custom fused kernels. These features are designed to support efficient quantized backpropagation and minimize memory bandwidth overhead. Across various tasks like customization, reinforcement learning, and distillation, FourTune consistently matches the quality of full-precision fine-tuning. For instance, on FLUX.1-dev (12B), it reduced memory overhead by 2.25 times and boosted training throughput by 2.27 times compared to BF16 LoRA, marking a significant leap in efficiency.