Bayesian Pooling Offers Limited Gains in BCI Classification

Brain-computer interfaces (BCIs) aim for calibration-free operation, but traditional classifiers often prioritize discrimination over the calibration of predicted probabilities, which is crucial given the nonstationary nature of EEG signals. This research investigated whether Bayesian complete-pooling models could offer advantages over frequentist methods in cross-subject motor imagery EEG classification. The study involved 20 datasets, comparing six frequentist pipelines with analogous Bayesian pipelines, all sharing identical feature engineering and fit via Markov chain Monte Carlo. The primary evaluation metric was the Brier score, decomposed into reliability and resolution, alongside AUROC for discrimination and Shannon entropy for sharpness. While Bayesian complete-pooling showed statistically significant improvements in reliability and increased predictive uncertainty (lower sharpness), these gains were not considered practically significant. No significant differences were observed in Brier score, resolution, or discrimination. Furthermore, the Bayesian pipelines were found to be approximately thirteen times more energy-intensive than their frequentist counterparts, though the absolute energy cost remained modest. The findings suggest that Bayesian complete-pooling alone provides limited practical benefits for this specific BCI application, pointing towards partial-pooling across subjects and sessions as a more promising avenue for future research.