New Theory Explains Random Forest Ensemble Size Tuning Dynamics

Researchers have introduced a theoretical framework to understand the behavior of Random Forest ensemble-size selection, specifically focusing on plateau-based tuning methods. These methods adjust the number of trees by comparing out-of-bag scores at different tree counts. The new theory models this process not as a deterministic convergence, but as a stochastic birth-death Markov chain, where the optimal ensemble size fluctuates around a stationary distribution. The theory provides equilibrium equations for the update rules, showing that the stationary center of the ensemble size scales inversely with the square of a small parameter. It also characterizes the stationary spread, indicating that the variance scales even more rapidly. These findings offer a deeper, mechanistic interpretation of how plateau-based tuning operates, moving beyond empirical observations.