Finding Multivariate Splits in Decision Trees Using Function Optimization

Book One

Volume

Proceedings of the AAAI Conference on Artificial Intelligence, 12

Student Abstracts

We present a new method for top-down induction of decision trees (TDIDT) with multivariate binary splits at the nodes. The primary contribution of this work is a new splitting criterion called soft entropy, which is continuous and differentiable with respect to the parameters of the splitting function. Using simple gradient descent to find multivariate splits and a novel pruning technique, our TDIDT-SEH (Soft Entropy Hyperplanes) algorithm is able to learn very small trees with better accuracy than competing learning algorithms on most datasets examined.