Leakage-Free Ensemble Boosts Multiclass Classification Accuracy.

Multiclass classification remains a significant challenge due to issues like high inter-class similarity, imbalanced datasets, and varying data distributions. While rule-based classifiers like XGBoost excel with structured features, they struggle with smooth functional relationships. Conversely, neural networks can capture complex non-linear interactions but often face overfitting and generalization problems. To address these limitations, a new framework called LFS-FRAME (Leakage-Free Stacked Ensemble) has been introduced. LFS-FRAME integrates functional learning, specifically using Kolmogorov-Arnold Networks (KAN), with rule-based learning via XGBoost. A key innovation is its strict out-of-fold stacking strategy, which creates unbiased meta-features by ensuring complete separation between training and validation data. This prevents performance leakage, a common pitfall in ensemble methods. By learning over the probabilistic outputs of these diverse base learners, the meta-classifier effectively combines global functional patterns with sharp decision boundaries. Experimental evaluations on various multiclass datasets demonstrated that LFS-FRAME significantly improves performance metrics, achieving an overall accuracy of 89.85% for major families and 81.74% for sub-families, outperforming strong single-model baselines. These results underscore the effectiveness of this leakage-free, hybrid stacking approach for reliable and generalizable multiclass classification.