Tuning Classifier Alone Boosts Semi-Supervised Security AI

In security classification, labeled data is often scarce, making semi-supervised learning (SSL) a valuable technique for propagating labels from small labeled datasets to larger unlabeled pools. However, SSL is frequently used as a black box, with default parameters, fixed classifiers, and inadequate handling of class imbalance caused by pseudo-labels. Recent studies have reported substantial performance improvements from optimizing SSL pipelines through joint search or per-component tuning, but it has been unclear whether these gains stem from complex SSL-classifier interactions or simply from better tuning of the underlying classifier. To clarify this, the researchers developed SemiScope, an analysis instrument designed to disentangle these effects for binary tabular security data using classical SSL and tree-based classifiers. SemiScope employs Bayesian Optimization to jointly tune SSL settings, confidence filtering, oversampling, and the classifier itself. A key control, "Tuned-Clf," fixes SSL to defaults but allocates the same computational budget for classifier hyperparameter optimization and validation-set threshold tuning as SemiScope. The results show that while SemiScope significantly outperforms all default SSL baselines, the "Tuned-Clf" approach achieves statistically equivalent performance on four out of five datasets under an equal budget. Classifier hyperparameter optimization alone recovered a median of 86% of the gains seen from the full SemiScope pipeline over a default Self-Training (ST) with Random Forest (RF) baseline. The conclusion is that a simpler recipe—using Self-Training, tuning the classifier with Bayesian Optimization, and tuning the decision threshold on validation data—is often sufficient, reaching near supervised performance with fewer labels.