The Need for Context in Multi-Modal Interfaces

Elise H. Turner, Roy M. Turner, Charles Grunden, Jason Mailman, Mark Neale, and John Phelps

Context is important for AI applications that interact with users. This is true both for natural language interfaces as well as for multi-modal interfaces. In this paper, we consider the kinds of contexts that are important in a multi-modal interface combining natural language and graphical input to describe locations. The descriptions will then be converted into queries to a geographical database system. We have identified several kinds of contexts in our preliminary study. We describe them and consider how each affects the system’s interpretation of user input. Plans for future work on the project are also presented, both for implementation and for empirical studies.


This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.