Dynamic-in-Few-Step Accelerates Video Diffusion Models by 30x

Video Diffusion Models (VDMs) are known for their high-quality generation but demand significant computational resources. While existing few-step distillation methods offer some acceleration, they typically use a static model architecture, failing to account for the varying computational needs across different denoising stages. This research proposes a novel post-training acceleration framework called "Dynamic-in-Few-Step" to address this limitation. The framework integrates dynamic structural sparsification directly into the distillation process. Instead of applying post-hoc compression, it jointly optimizes denoising steps and structured model sparsity, effectively transforming a pre-trained VDM into a compact, step-specific Mixture-of-Models (MoM). To ensure stable training during this joint optimization, a Progressive Training Strategy combined with an Output Rollout Mechanism is introduced. The method is highly effective, achieving a 1.2x wall-clock gain on top of 4-step distillation and an overall 30x speedup compared to a 50-step teacher model, all while preserving competitive generation quality.