Examining Extended Dynamic Scripting in a Tactical Game Framework

Dynamic scripting is a reinforcement learning algorithm designed specifically to learn appropriate tactics for an agent in a modern computer game, such as Neverwinter Nights. This reinforcement learning algorithm has previously been extended to support the automatic construction of new abstract states to improve its context sensitivity and integrated with a graphical behavior modeling architecture to allow for hierarchical dynamic scripting and task decomposition. In this paper, we describe a tactical abstract game representation language that was designed specifically to make it easier to define abstract games that include the large amount of uncertainty found in modern computer games. We then use this framework to examine the effectiveness of the extended version of the dynamic scripting algorithm, using Q-learning and the original dynamic scripting algorithms as benchmarks. Results and discussion are provided for three different abstract games: one based on combat in role-playing games and two based on different aspects of real-time strategy games.