Monte Carlo Tree Search in Continuous Spaces Using Voronoi Optimistic Optimization with Regret Bounds

Authors

  • Beomjoon Kim MIT
  • Kyungjae Lee Seoul National University
  • Sungbin Lim Kakao Brain
  • Leslie Kaelbling MIT
  • Tomas Lozano-Perez MIT

DOI:

https://doi.org/10.1609/aaai.v34i06.6546

Abstract

Many important applications, including robotics, data-center management, and process control, require planning action sequences in domains with continuous state and action spaces and discontinuous objective functions. Monte Carlo tree search (MCTS) is an effective strategy for planning in discrete action spaces. We provide a novel MCTS algorithm (voot) for deterministic environments with continuous action spaces, which, in turn, is based on a novel black-box function-optimization algorithm (voo) to efficiently sample actions. The voo algorithm uses Voronoi partitioning to guide sampling, and is particularly efficient in high-dimensional spaces. The voot algorithm has an instance of voo at each node in the tree. We provide regret bounds for both algorithms and demonstrate their empirical effectiveness in several high-dimensional problems including two difficult robotics planning problems.

Downloads

Published

2020-04-03

How to Cite

Kim, B., Lee, K., Lim, S., Kaelbling, L., & Lozano-Perez, T. (2020). Monte Carlo Tree Search in Continuous Spaces Using Voronoi Optimistic Optimization with Regret Bounds. Proceedings of the AAAI Conference on Artificial Intelligence, 34(06), 9916-9924. https://doi.org/10.1609/aaai.v34i06.6546

Issue

Section

AAAI Technical Track: Planning, Routing, and Scheduling