Efficient Planning and Tracking in POMDPs with Large Observation Spaces

Contents

Planning in partially observable MDPs is computationally limited by the size of the state, action and observation spaces. While many techniques have been proposed to deal with large state and action spaces, the question of automatically finding good low-dimensional observation spaces has not been explored as thoroughly. We show that two different reduction algorithms, one based on clustering and the other on a modified principal component analysis, can be applied directly to the observation probabilities to create a reduced feature observation matrix. We apply these techniques to a real-world dialogue management problem, and show that fast and accurate tracking and planning can be achieved using the reduced observation spaces.