The Need for Context in Multi-Modal Interfaces

Contents

Context is important for AI applications that interact with users. This is true both for natural language interfaces as well as for multi-modal interfaces. In this paper, we consider the kinds of contexts that are important in a multi-modal interface combining natural language and graphical input to describe locations. The descriptions will then be converted into queries to a geographical database system. We have identified several kinds of contexts in our preliminary study. We describe them and consider how each affects the system’s interpretation of user input. Plans for future work on the project are also presented, both for implementation and for empirical studies.