Verification and Validation: A Case Study of a Decision Support System

Proceedings of the Artificial Intelligence and Manufacturing Research Planning Workshop, 1996

Volume

Proceedings of the Artificial Intelligence and Manufacturing Research Planning Workshop, 1996

Contents

In 1985 we started building a decision support system to help speech clinicians diagnose small children who have begun to stutter. This paper describes the verification and validation of that system: (1) having an expert use and evaluate it, (2) running test cases, (3) developing a program to detect redundant rules, (4) using the Analytic Hierarchy Process, (5) running a program that checks a knowledge base for consistency and completeness, (6) having five experts independently critique the system, (7) obtaining diagnoses of stuttering from these five experts derived from repom of children who had been evaluated for possible stuttering problems, (8) using the system to expose missing and ambiguous information in 30 clinical reports, and (9) analyzing the di~on and bias of six experts and the decision support system in diagnosing stuttering. When using the final system, three clinicians with widely differing backgrounds woduced diagnostic opinions that evidence little variability and were indistinguishable from those of a panel of five experienced clinicians.