AAAI Publications, Sixth AAAI Conference on Human Computation and Crowdsourcing

Font Size: 
Utilizing Crowdsourced Asynchronous Chat for Efficient Collection of Dialogue Dataset
Kazushi Ikeda, Keiichiro Hoashi

Last modified: 2018-06-15

Abstract


In this paper, we design a crowd-powered system to efficiently collect data for training dialogue systems. Conventional systems assign dialogue roles to a pair of crowd workers, and record their interaction on an online chat. In this framework, the pair is required to work simultaneously, and one worker must wait for the other when he/she is writing a message, which decreases work efficiency. Our proposed system allows multiple workers to create dialogues in an asynchronous manner, which relieves workers from time restrictions. We have conducted an experiment using our system on a crowdsourcing platform to evaluate the efficiency and the quality of dialogue collection. Results show that our system can reduce the necessary time to input a message by 68% while maintaining quality.

Keywords


conversational agent; dialogue data collection; efficiency; crowdsourcing

Full Text: PDF