Options of Interest: Temporal Abstraction with Interest Functions

Authors

  • Khimya Khetarpal McGill University
  • Martin Klissarov McGill University
  • Maxime Chevalier-Boisvert Universite de Montreal
  • Pierre-Luc Bacon Stanford University
  • Doina Precup McGill University

DOI:

https://doi.org/10.1609/aaai.v34i04.5871

Abstract

Temporal abstraction refers to the ability of an agent to use behaviours of controllers which act for a limited, variable amount of time. The options framework describes such behaviours as consisting of a subset of states in which they can initiate, an internal policy and a stochastic termination condition. However, much of the subsequent work on option discovery has ignored the initiation set, because of difficulty in learning it from data. We provide a generalization of initiation sets suitable for general function approximation, by defining an interest function associated with an option. We derive a gradient-based learning algorithm for interest functions, leading to a new interest-option-critic architecture. We investigate how interest functions can be leveraged to learn interpretable and reusable temporal abstractions. We demonstrate the efficacy of the proposed approach through quantitative and qualitative results, in both discrete and continuous environments.

Downloads

Published

2020-04-03

How to Cite

Khetarpal, K., Klissarov, M., Chevalier-Boisvert, M., Bacon, P.-L., & Precup, D. (2020). Options of Interest: Temporal Abstraction with Interest Functions. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 4444-4451. https://doi.org/10.1609/aaai.v34i04.5871

Issue

Section

AAAI Technical Track: Machine Learning