New SVM Boosts Robustness and Feature Selection with RoBoSS Loss

Real-world datasets frequently present challenges such as redundant features, label noise, and feature noise, which can severely hinder the efficiency and accuracy of machine learning models. Traditional Support Vector Machines (SVMs), particularly those using l2-norm penalties and hinge loss, are often sensitive to noise and lack inherent feature selection capabilities. To overcome these limitations, researchers propose a novel approach: the asymmetric, robust, bounded, sparse, and smooth (aR) loss function, integrated into an l1-norm penalized geometric twin SVM (aRSGTSVM). The l1-norm penalty inherently enables significant feature selection, addressing the issue of redundant features. The innovative aR loss function is designed to effectively reduce the impact of label noise and substantially improve stability against resampling noise, specifically zero-mean feature noise near decision boundaries. A statistical analysis using influence functions further confirms the robustness of aRSGTSVM. A fast proximal gradient descent algorithm is developed to handle the non-convex and non-smooth optimization. Experimental results on synthetic and UCI datasets, along with an application to stock market index tracking, demonstrate its superior performance compared to existing state-of-the-art methods.