Quantitative Performance Prediction for Rule-based Expert Systems

Contents

Often a rule-based system is tested by checking its performance on a number of test cases with known solutions, modifying the system until it gives the correct results for all or a sufficiently high proportion of the test cases. However, the performance on the test cases may not accurately predict performance of the system in actual use. We present a method for making a more accurate performance prediction based on the performance on test cases, coverage of the rule-base by the test cases, how representative the test data is of the population on which the rule-base will be used, and the likelihood of occurrence of different kinds of test cases in the larger population.