Adaptive Training Improves Risk-Aware Q-Learning for Finance.

Risk-aware Q-learning (RaQL), particularly with Conditional Value-at-Risk (CVaR), is a powerful tool for dynamic risk management but often struggles with stability and sample efficiency under limited training budgets. Fixed hyperparameters in its inner-loop can lead to unstable value estimates and inefficient use of data. This research proposes an adaptive training controller specifically designed for CVaR RaQL. The controller introduces six coordinated mechanisms, including dynamic inner-step sizing, synchronized decay, early correction for VaR-like variables, and data-driven calibration. These mechanisms optimize the training process without altering the core risk objective. Evaluated on a daily Bitcoin trading task, the adaptive controller dramatically improved performance. It reduced the mean empirical CVaR Bellman residual by approximately 85% compared to a fixed-parameter baseline. The learned policy achieved a Sharpe ratio of 0.9281 with significantly lower volatility and maximum drawdown, demonstrating enhanced reliability and risk-adjusted returns in financial applications.