Conditional Inference Forests Excel in Feature Selection and Ranking.

Conditional inference trees (CIT) and conditional inference forests (CIF) are machine learning methods designed to mitigate split-selection bias by testing features before determining split thresholds. While these methods can be computationally intensive due to repeated permutation tests and threshold searches, this research investigates their utility as top-k feature-ranking tools for downstream predictive tasks. The study utilized real-world data benchmarks, runtime ablations, and synthetic feature-recovery experiments. It found that CIF ranks highly among other classification and regression methods across numerous datasets. The research also explored runtime optimizations, indicating that adaptive stopping and the number of thresholds searched have the most significant impact on fitting time, yet their adjustments resulted in minimal changes to downstream prediction scores. This suggests that CIF offers a robust and efficient approach for feature selection, even with certain computational shortcuts.