Visualizing Bagged Decision Trees

All Contents

We present a visual tablet for expolring the nature of a bagged decision tree (Breiman 1984). Aggregating classifiers over bootstrap datasets (bagging) can result in greatly improved prediction accuracy. Bagging is motivated as a variance reduction technique, but it is considered a black box with respect to interpretation. Current research seeking to explain why bagging works has focused on different bias/variance decompositions of prediction error. We show that bagging’s complexity can be better understood by a simple graphical technique that allows visualizing the bagged decision boundary in low dimensional situations. We then show that bagging can be heuristically motivated as a method to enhance local adaptivity of the boundary. Simulated examples are presented to illustrate the technique.