Performance Analysis of Planning Portfolios

In recent years the concept of sequential portfolio has become an important topic to improve the performance of modern problem solvers, such as SAT engines or planners. The PbP planner and more recently Fast Downward Stone Soup are successful approaches in Automated Planning that follow this trend. However, neither a theoretical analysis nor formal definitions about sequential portfolios have been described. In this paper, we focus on studying how to evaluate the performance of planners defining a baseline for a set of problems. We present a general method based on Mixed-Integer Programming to define the baseline for a training data set. In addition to prior work, we also introduce a short empirical analysis of the utility of training problems to configure sequential portfolios.