Naturalistic Distributed Experimentation as a Source of New Insight

Human performance experiments are often conducted online with the help of paid crowdworkers and citizen scientists. This approach produces reliable data, but there are concerns that the inevitable loss of control that accompanies online experimentation might confound results. Researchers have therefore spent time considering how to regain control and mitigate the effects of confounds. In this abstract we argue that confounding factors in online work can be put to novel use, giving us insight into research questions we might otherwise be unable to answer.