A Proposed Model for Effective Verification of Natural Language Generation Systems

Natural language processing (NLP) research is carried out in areas such as speech recognition, natural language understanding (NLU), natural language generation (NLG), speech synthesis, information retrieval, information extraction, and inference. NLP components are being incorporated into a variety of systems, and NLP methods are being used in new application areas. There is increasing interest in dialogue systems and language generation systems (e.g. the ELVIS system for voice access to email and voicemail, the embodied conversational agent REA, text analysis systems, summarization systems, information extraction systems, and text-to-speech systems. In practice these activities require building systems that model human activities in various language processing tasks. Therefore, we can view language processing systems as intelligent systems. These uses, furthermore, increase the need for thorough testing of NLP systems and individual NLP components that are embedded in larger systems. Language processing researchers have not generally carried out the sorts of verification and validation activities that are typically attempted in the intelligent systems research area. The research presented here is part of a larger project that is considering how verification and validation can be carried out for language processing systems in different application areas.