Cross-Regime Bayesian Optimization Boosts Algorithmic Trading Signals

Algorithmic trading is a multi-billion dollar market where even small improvements in signal robustness can yield substantial financial gains. Current evaluations of equity prediction models often overlook the explicit targeting of regime robustness during hyperparameter selection, which is crucial given varying market conditions. This research addresses this by training five different model classes on daily data from approximately 300 large-cap US equities over eleven years. A key innovation is the use of Bayesian optimization configured to specifically target trading performance across three statistically distinct market regimes. This regime-robust hyperparameter selection proved vital for out-of-sample generalization, maintaining signal precision above a random baseline throughout the test period. While no single tabular deep learning architecture surpassed gradient-boosted trees (XGBoost), a hybrid ensemble combining XGBoost and TabNet through rank aggregation achieved remarkable results: an annualized return of 51.26%, a Sharpe ratio of 2.44, and a statistically significant CAPM alpha of 0.423. A near-zero beta indicates this outperformance stems from superior stock selection rather than market exposure. The study also notes that alternative data plays a secondary role once technical and fundamental features are included, contributing more to short-side predictions. An interactive application allows real-time exploration of these findings.