Experiences in Improving the State of the Practice in Verification and Validation of Knowledge-based Systems

Contents

Knowledge-based systems (KBS) are in general use in a wide variety of domains. As reliance on these types systems grows, the need to assess their quality and validity reaches critical importance. As with any software, the reliability of a KBS can be directly attributed to the application of disciplined programming and testing practices throughout the development life-cycle. However, there are some essential differences between conventional (i.e., procedural) software and knowledge-based systems, both in construction and use. The identification of these differences affect the verification and validation process and the development of techniques to handle them is the basis of considerable on-going research in this field. For the past three years IBM (Federal Systems Company - Houston) and the Software Technology Branch ( STB) of NASA~Johnson Space Center have been working to improve the "’state of the practice" in verification and validation of Knowledge-based systems. To date, the primary accomplishment has been the development and teaching of a four-day workshop on verification and validation of knowledge-based systems. With the hope of improving the impact of these workshops, we are now working directly with NASA KBS projects to employ concepts taught in the workshop. This paper describes two projects currently being assisted as part of this effort. In addition to describing each project, this paper describes problems encountered and solutions proposed in each case.