Universal Value Iteration Networks: When Spatially-Invariant Is Not Universal

Authors

  • Li Zhang Beijing Institute of Technology
  • Xin Li Beijing Institute of Technology
  • Sen Chen Beijing Institute of Technology
  • Hongyu Zang Beijing Institute of Technology
  • Jie Huang Beijing Institute of Technology
  • Mingzhong Wang University of the Sunshine Coast

DOI:

https://doi.org/10.1609/aaai.v34i04.6157

Abstract

In this paper, we first formally define the problem set of spatially invariant Markov Decision Processes (MDPs), and show that Value Iteration Networks (VIN) and its extensions are computationally bounded to it due to the use of the convolution kernel. To generalize VIN to spatially variant MDPs, we propose Universal Value Iteration Networks (UVIN). In comparison with VIN, UVIN automatically learns a flexible but compact network structure to encode the transition dynamics of the problems and support the differentiable planning module. We evaluate UVIN with both spatially invariant and spatially variant tasks, including navigation in regular maze, chessboard maze, and Mars, and Minecraft item syntheses. Results show that UVIN can achieve similar performance as VIN and its extensions on spatially invariant tasks, and significantly outperforms other models on more general problems.

Downloads

Published

2020-04-03

How to Cite

Zhang, L., Li, X., Chen, S., Zang, H., Huang, J., & Wang, M. (2020). Universal Value Iteration Networks: When Spatially-Invariant Is Not Universal. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 6778-6785. https://doi.org/10.1609/aaai.v34i04.6157

Issue

Section

AAAI Technical Track: Machine Learning