Counterfactual Data Augmentation Boosts Regression Model Accuracy.

Data-driven modeling in real-world regression tasks frequently struggles with limited training data, high collection costs, and noisy observations. Drawing inspiration from the success of data augmentation in computer vision and natural language processing, this research introduces Counterfactual Residual Data Augmentation (CRDA), a novel technique specifically designed for tabular regression. The core insight behind CRDA is that once a regressor has captured the systematic patterns in the data, the remaining noise can be treated as an invariant residual that remains stable even when small, carefully chosen perturbations are applied to features. By leveraging this residual invariance, CRDA can generate new, yet realistic, training samples, effectively expanding the dataset without requiring additional real-world data collection. The method is model-agnostic, making it broadly applicable to various regressors. Extensive experiments across diverse benchmark datasets demonstrated significant performance improvements: CRDA reduced the Mean Squared Error (MSE) for MLP Regressors by an average of 22.9% and for XGBoost Regressors by 6.4%. It consistently outperformed existing state-of-the-art data generators and augmentation techniques in MSE reduction, offering a simple and efficient solution for noise-prone, small-sample regression problems.