Multistage Defer Trees Enhance Model Interpretability and Accuracy

The field of machine learning often faces a trade-off between model accuracy and interpretability. While individual decision trees offer high interpretability, complex ensemble models typically achieve superior accuracy, albeit at the cost of transparency. This new research aims to bridge this gap by proposing Multistage Defer Trees. This novel model class consists of a sequence of sparse decision trees. Each tree in the sequence is designed to classify the majority of data points, providing an interpretable decision path. For a small proportion of samples that a tree cannot confidently classify, it "defers" them to the next tree in the sequence. Ultimately, any remaining highly complex or uncertain cases are passed to a more powerful, but less interpretable, black-box model. The researchers demonstrate that this method can achieve performance comparable to complex tree-based ensembles. Crucially, it does so while ensuring that most samples are processed by only one or a few simple, interpretable decision trees. This approach expands the accuracy-interpretability frontier, offering a practical solution for scenarios where high accuracy is needed but transparency for most decisions is also paramount.