Quantifying the Impact of Learning Algorithm Parameter Tuning

Proceedings of the AAAI Conference on Artificial Intelligence, 21

Volume

Technical Papers

Machine Learning

The impact of learning algorithm optimization by means of parameter tuning is studied. To do this, two quality attributes, sensitivity and classification performance, are investigated, and two metrics for quantifying each of these attributes are suggested. Using these metrics, a systematic comparison has been performed between four induction algorithms on eight data sets. The results indicate that parameter tuning is often more important than the choice of algorithm and there does not seem to be a trade-off between the two quality attributes. Moreover, the study provides quantitative support to the assertion that some algorithms are more robust than others with respect to parameter configuration. Finally, it is briefly described how the quality attributes and their metrics could be used for algorithm selection in a systematic way.