Reflection in Action: Model-Based Self-Adaptation in Game Playing Agents

Contents

Computer war strategy games offer a challenging domain for AI techniques for learning because they involve multiple players, the world in the games is only partially observable and the state space is extremely large. Model-based reflection and self-adaptation is one method for learning in such a complex domain. In this method, the game-playing agent contains a model of its own reasoning processes. When the agent fails to win a game, it uses its self-model and (possibly) traces of its execution to analyze the failure and modify its knowledge and reasoning accordingly. In this paper, we describe an experimental investigation of model-based reflection and self-adaptation for a specific task (defending a city) in a computer war strategy game called Civilization. Our results indicate that at least for limited tasks, model-based reflection enables effective learning, and further, when traces are used in conjunction with the model, the effectiveness of learning appears to increase with the size of the trace.