AAAI Publications, Eleventh Artificial Intelligence and Interactive Digital Entertainment Conference

Font Size: 
A Benchmark for StarCraft Intelligent Agents
Alberto Uriarte, Santiago Ontañón

Last modified: 2015-09-23

Abstract


The problem of comparing the performance of different Real-Time Strategy (RTS) Intelligent Agents (IA) is non-trivial. And often different research groups employ different testing methodologies designed to test specific aspects of the agents. However, the lack of a standard process to evaluate and compare different methods in the same context makes progress assessment difficult. In order to address this problem, this paper presents a set of benchmark scenarios and metrics aimed at evaluating the performance of different techniques or agents for the RTS game StarCraft. We used these scenarios to compare the performance of a collection of bots participating in recent StarCraft AI (Artificial Intelligence) competitions to illustrate the usefulness of our proposed benchmarks.

Keywords


RTS; real-time strategy games; benchmark; StarCraft; bot; AI; BWAPI

Full Text: PDF