Be Inaccurate but Don’t Be Indecisive: How Error Distribution Can Affect User Experience

Authors

  • Rafael R. Padovani Universidade Federal de Viçosa
  • Lucas N. Ferreira University of California, Santa Cruz
  • Levi H. S. Lelis Universidade Federal de Viçosa

DOI:

https://doi.org/10.1609/aaai.v33i01.33012604

Abstract

System accuracy is a crucial factor influencing user experience in intelligent interactive systems. Although accuracy is known to be important, little is known about the role of the system’s error distribution in user experience. In this paper we study, in the context of background music selection for tabletop games, how the error distribution of an intelligent system affects the user’s perceived experience. In particular, we show that supervised learning algorithms that solely optimize for prediction accuracy can make the system “indecisive”. That is, it can make the system’s errors sparsely distributed throughout the game session. We hypothesize that sparsely distributed errors can harm the users’ perceived experience and it is preferable to use a model that is somewhat inaccurate but decisive, than a model that is accurate but often indecisive. In order to test our hypothesis we introduce an ensemble approach with a restrictive voting rule that instead of erring sparsely through time, it errs consistently for a period of time. A user study in which people watched videos of Dungeons and Dragons sessions supports our hypothesis.

Downloads

Published

2019-07-17

How to Cite

Padovani, R. R., Ferreira, L. N., & Lelis, L. H. S. (2019). Be Inaccurate but Don’t Be Indecisive: How Error Distribution Can Affect User Experience. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 2604-2611. https://doi.org/10.1609/aaai.v33i01.33012604

Issue

Section

AAAI Technical Track: Humans and AI