Grounding Language in Spatial Routines

Control Mechanisms for Spatial Knowledge Processing in Cognitive / Intelligent Systems

Volume

Papers from the 2007 AAAI Spring Symposium

Contents

This paper describes a spatial language understanding system based on a lexicon of words defined in terms of spatial routines. A spatial routine is a script composed from a set of primitive operations on sensor data, analogous to Ullman's visual routines. We hypothesize that a set of primitives that underlie spatial language could be used to succinctly express the meaning of spatial terms in a way that can be used to interpret natural language commands. This hypothesis is tested by using spatial routines to build a natural language interface to a real-time strategy game, in which a player controls an army of units in a battle. Spatial routines are used as a top down control mechanism for the system, providing a mapping from natural language commands to system behavior. Although the current implementation has significant limitations as revealed in an initial evaluation, we believe that the concept of spatial routines holds promise as a way to ground spatial language semantics in terms of embodied sensory-motor interaction.