New Theory Quantifies Myopic Active Learning Risk

Active learning is a fundamental area of machine learning focused on intelligently selecting data points for observation to optimize learning. A common heuristic in this field is the greedy algorithm, which is equivalent to myopic Bayesian active learning for linear regression. This approach simplifies long-term planning by making the locally optimal choice at each step. Despite its widespread use, a formal understanding of the greedy algorithm's performance, particularly its risk, has been lacking. This research provides a significant theoretical contribution by proving the first approximation ratio for the risk of this algorithm. The derived approximation ratio is shown to be tight up to an absolute constant and is directly proportional to the maximum initial leverage score (MILS). MILS is introduced as a new, fundamental quantity that dictates the greedy algorithm's effectiveness. The paper illustrates these theoretical findings with simple numerical simulations, offering a clearer understanding of when and why myopic active learning performs well or poorly.