K-ABENA Optimizes Backpropagation by Excluding Low-Loss Samples

Researchers have introduced K-ABENA, a novel framework designed to optimize the backpropagation process in neural network training. This method significantly reduces per-iteration training costs by selectively excluding a portion of "minor" observations, specifically those with low loss, from the backward pass computation. The core innovation lies in its compensated loss-based sample exclusion, which employs a defensive-mixture sampling design combined with Horvitz-Thompson inverse-probability reweighting. This approach ensures an unbiased gradient estimator, a critical improvement over previous uncompensated methods that suffered from significant bias and poor performance, especially with class imbalance. Empirical validation on real datasets like Breast Cancer and Digits shows K-ABENA achieving statistically indistinguishable performance from full-batch SGD while saving 28-54% of per-epoch gradient computation. This advancement offers a practical way to accelerate deep learning training without sacrificing accuracy.