Uncertainty guided pruning of classification model tree

A model tree is a hybrid learning algorithm that integrates decision trees and embedding models, with simple structure and high interpretability. However, all the existing works on model trees neglect to estimate the reliability of the output. This estimate not only plays an important role in model selection but also provides an effective guide to the optimization of model tree performance. This work first introduces the output uncertainty of the embedding model into the model tree building process. Specifically, we model the tree based on an expanded post-pruning rule which introduces output uncertainty. At the same time, we include an error estimation term for the embedded model, in which the output uncertainty gives reverse guidance to the model performance. Besides, the proposed optimization rules can be extended as a supplementary condition to any existing post-pruning method. The uncertainty guided model tree is introduced and presented in detail by extending two post-pruning methods which are generally expected to have higher accuracy. Experiments on 12 benchmark data sets demonstrate the superiority of the proposed method.