Fourier Analysis Explains Partial Data Augmentation's Effectiveness

Data augmentation is a widely used, model-agnostic technique that leverages known invariances in learning problems by expanding training datasets with transformed copies of existing samples. While effective, applying full group-sized augmentation can become computationally prohibitive when the transformation group is large. This raises a critical question: can partial data augmentation deliver the same statistical advantages as full augmentation in terms of generalization and sample complexity? A new framework, utilizing Fourier analysis and the representation theory of finite groups, investigates this question. The study demonstrates that for a broad range of classical learning problems, partial data augmentation—where only a randomly sampled subset of group elements is used—can achieve the same minimax rates as full augmentation. This holds true with an approximation error that diminishes as the size of the sampled subset increases. These findings provide a theoretical basis for why partial augmentation can retain the statistical benefits of full augmentation, even though it only approximately enforces symmetry. The research also presents a complementary impossibility result, showing that enforcing exact invariance through data augmentation requires averaging over the entire group, and cannot be achieved by any strict subset if the hypothesis space is sufficiently expressive. Together, these results offer a unified perspective on the trade-offs between full and partial data augmentation and exact versus approximate symmetry enforcement.