Improving Dialogue Annotation Reliability

Applying Machine Learning to Discourse Processing

Volume

Papers from the 1998 AAAI Spring Symposium

Contents

Data collection and annotation represent the most significant portion of the effort involved in building statistical models of dialogue. We must be able to annotate reliably if there is any hope of building predictive statistical models. This paper describes the annotation process being used to tag the TRAINS-93 corpus with dialogue acts, and measures we have taken to improve inter-annotator reliability.