Applying Reliability Engineering to Expert Systems

May 1999

Proceedings of the Twelfth International Florida Artificial Intelligence Research Society Conference (FLAIRS 1999)

Volume

Proceedings of the Twelfth International Florida Artificial Intelligence Research Society Conference (FLAIRS 1999)

All Papers

Often a rule-based system is tested by checking its performance on a number of test cases with known solutions, modifying the system until it gives the correct results for all or a sufficiently high proportion of the test cases. However, the performance on the test cases may not accurately predict performance of the system in actual use. In this paper we discuss why this testing method does not give an accurate reliability prediction. We then propose a method for reliability prediction based on coverage data, attained during testing, and an operational profile of the expert system. This extension of software reliability engineering methods into the expert systems realm can lead to better systems as well as a more accurate prediction of system behavior in actual use.