Automatic Sample-by-Sample Model Selection Between Two Off-the-Shelf Classifiers

Proceedings of the AAAI Conference on Artificial Intelligence, 16

Volume

Proceedings of the AAAI Conference on Artificial Intelligence, 16

Student Abstracts

If one could predict which of two classifiers will correctly classify a particular sample, then one could use the better classifier. Continuing this selection process throughout the data set should result in improved accuracy over either classifier alone. Fortunately, scalar measures which relate to the degree of confidence that we have in a classification can be computed for most common classifiers. Some examples of confidence measures are distance from a linear discriminant separating plane, distance to the nearest neighbor, distance to the nearest unlike neighbor, and distance to the center of correctly classified training data. We propose to apply discriminant analysis to the confidence measures, producing a rule which determines when one classifier is expected to be more accurate than the other.