Second Opinion: Supporting Last-Mile Person Identification with Crowdsourcing and Face Recognition

2019-10-21

Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 7

Volume

Vol. 7 (2019): Proceedings of the Seventh AAAI Conference on Human Computation and Crowdsourcing

Technical Papers

As AI-based face recognition technologies are increasingly adopted for high-stakes applications like locating suspected criminals, public concerns about the accuracy of these technologies have grown as well. These technologies often present a human expert with a shortlist of high-confidence candidate faces from which the expert must select correct match(es) while avoiding false positives, which we term the “last-mile problem.” We propose Second Opinion, a web-based software tool that employs a novel crowdsourcing workflow inspired by cognitive psychology, seed-gather-analyze, to assist experts in solving the last-mile problem. We evaluated Second Opinion with a mixed-methods lab study involving 10 experts and 300 crowd workers who collaborate to identify people in historical photos. We found that crowds can eliminate 75% of false positives from the highest-confidence candidates suggested by face recognition, and that experts were enthusiastic about using Second Opinion in their work. We also discuss broader implications for crowd–AI interaction and crowdsourced person identification.

10.1609/hcomp.v7i1.5272