Distributed Sketching Improves OLS Regression on Partitioned Data

The paper explores a technique called distributed sketching, applied to Ordinary Least Squares (OLS) regression. Unlike previous methods that sketch an entire dataset, this research focuses on sketching smaller, partitioned subsets of data across multiple machines. The goal is to construct individual OLS estimators from these sketches and then average them. The study specifically analyzes the fixed design setting, providing a precise characterization of the excess loss incurred by the averaged OLS estimator. This analysis reveals that the loss achieved by sketching on partitioned subsets is comparable to that of sketching the entire dataset. This comparability holds particularly true when the divergence or differences among the covariances of the individual data subsets are minimal. The findings suggest a computationally efficient approach for large-scale OLS regression by distributing the sketching process.