The ActiveCrowdToolkit: An Open-Source Tool for Benchmarking Active Learning Algorithms for Crowdsourcing Research

2015-11-12

Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 3

Volume

Vol. 3 (2015): Third AAAI Conference on Human Computation and Crowdsourcing

Works in Progress

We present an open-source toolkit that allows the easy comparison of the performance of active learning methods over a series of datasets. The toolkit allows such strategies to be constructed by combining a judgement aggregation model, task selection method and worker selection method.The toolkit also provides a user interface which allows researchers to gain insight into worker performance and task classification at runtime.

10.1609/hcomp.v3i1.13256