Stefanie Brüninghaus and Kevin D. Ashley
Evaluation is an inevitable part of a research project. However, it is still an open issue how to evaluate Case-Based Reasoning systems for applications where the cases are text documents. In this context, three particular questions arise: (1) What performance measure to use, (2) what standard of comparison to apply, and (3) which instrinsic or extrinsic features of the system to focus on. For each of these issues, we elaborate alternatives and relevant factors that should be taken into consideration when conducting a formative evaluation of a textual CBR system. This discussion also considers work done in other projects, as well as in our own research.