Abstract:
The main aim of this paper is to suggest multi-criteria based metrics that can be used as comparators for an objective evaluation of data mining algorithms (DM-algorithms). Each DM-algorithm is characterized, generally, by some positive and negative properties, when it is applied to certain domains. Examples of properties are the accuracy rate, understandability, interpretability of the generated results and stability. Space and time complexity and maintenance costs can be considered as negative properties. By now there is no methodology to consider all of these properties, simultaneously, and use them for a comprehensive evaluation of DM-algorithms. Most of available studies in literature use only the accuracy rate as a unique criterion to compare the performance of DM-algorithms and ignore the other properties. Our suggested approach, however, can take into account all available positive and negative characteristics of DM-algorithms and can combine them to construct a unique evaluation metric. This new approach is based on DEA (Data Envelopment Analysis). We have applied this approach to evaluate 23 DM-algorithms in 22 domains. The results are analyzed and compared with the results of alternative approaches.