Extending Workers’ Attention Span Through Dummy Events

2016-11-03

Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 4

Volume

Vol. 4 (2016): Fourth AAAI Conference on Human Computation and Crowdsourcing

Full Papers

This paper studies a new paradigm for improving the attention span of workers in tasks that heavily rely on user's attention to the occurrence of rare events. Such tasks are highly common, ranging from crime monitoring to controlling autonomous complex machines, and many of them are ideal for crowdsourcing.  The underlying idea in our approach is to dynamically augment the task with some dummy (artificial) events at different times throughout the task, rewarding the worker upon identifying and reporting them.  This, as an alternative to the traditional approach of exclusively relying on rewarding the worker for successfully identifying the event of interest itself.  We propose three methods for timing the dummy events throughout the task. Two of these methods are static and determine the timing of the dummy events at random or uniformly throughout the task. The third method is dynamic and uses the identification (or misidentification) of dummy events as a signal for the worker's attention to the task, adjusting the rate of dummy events generation accordingly. We use extensive experimentation to compare the methods with the traditional approach of inducing attention through rewarding the identification of the event of interest and within the three. The analysis of the results indicates that with the use of dummy events a substantially more favorable tradeoff between the detection (of the event of interest) probability and the expected expense can be achieved, and that among the three proposed method the one that decides on dummy events on the fly is (by far) the best.

10.1609/hcomp.v4i1.13276