Universal Value Iteration Networks: When Spatially-Invariant Is Not Universal

  • Li Zhang Beijing Institute of Technology
  • Xin Li Beijing Institute of Technology
  • Sen Chen Beijing Institute of Technology
  • Hongyu Zang Beijing Institute of Technology
  • Jie Huang Beijing Institute of Technology
  • Mingzhong Wang University of the Sunshine Coast

Abstract

In this paper, we first formally define the problem set of spatially invariant Markov Decision Processes (MDPs), and show that Value Iteration Networks (VIN) and its extensions are computationally bounded to it due to the use of the convolution kernel. To generalize VIN to spatially variant MDPs, we propose Universal Value Iteration Networks (UVIN). In comparison with VIN, UVIN automatically learns a flexible but compact network structure to encode the transition dynamics of the problems and support the differentiable planning module. We evaluate UVIN with both spatially invariant and spatially variant tasks, including navigation in regular maze, chessboard maze, and Mars, and Minecraft item syntheses. Results show that UVIN can achieve similar performance as VIN and its extensions on spatially invariant tasks, and significantly outperforms other models on more general problems.

Published
2020-04-03
Section
AAAI Technical Track: Machine Learning