SCBoost Reduces Redundancy for Improved Boosting Performance.

Boosting frameworks, which sequentially fit models to residuals, often suffer from learner redundancy. This occurs because successive learners repeatedly address correlated error components, leading to inefficiencies. To counter this, a new framework called SCBoost has been proposed, which fundamentally shifts from traditional residual fitting to a more advanced concept of residual orthogonalization. SCBoost employs two complementary mechanisms to control redundancy. First, Spectral Residual Projection (SRP) ensures that each new residual target is projected onto the orthogonal complement of the historical prediction subspace. This forces subsequent learners to focus exclusively on capturing novel empirical innovations, preventing them from relearning already explained patterns. Second, Covariance-Regularized Weighting (CRW) optimizes the ensemble weights on a validation set while explicitly penalizing remaining correlations among learners, further enhancing efficiency. Theoretically, the researchers provide a geometric characterization demonstrating that SRP achieves an exact additive residual-energy decomposition. Experiments across ten benchmark datasets show that SCBoost delivers strong out-of-the-box performance, particularly in accuracy and F1 score, by offering a principled approach to explicit redundancy control in ensemble architectures.