Tuning the Diversity of Open-Ended Responses From the Crowd

2014-11-05

Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 2

Volume

Vol. 2 (2014): Second AAAI Conference on Human Computation and Crowdsourcing

Works in Progress

Crowdsourcing can solve problems beyond the reach of state-of-the-art fully automated systems. A common pattern found in many such systems is for the workers to discover, in parallel, a number of candidate solutions and then vote on the best one to pass forward, often within a fixed amount of time. We present the propose-vote-abstain mechanism for eliciting from crowd workers the proper balance between solution discovery and selection. Each crowd worker is given a choice among proposing an answer, voting among the answers proposed so far, or abstaining, i.e., doing nothing. When a stopping condition is reached, the mechanism returns the answer with the most votes. Workers are paid a base amount, with bonuses if they propose or vote for the winning answer.

10.1609/hcomp.v2i1.13199