Goal-Oriented Dialogue Policy Learning from Failures

Authors

  • Keting Lu University of Science and Technology of China
  • Shiqi Zhang State University of New York, Binghamton
  • Xiaoping Chen University of Science and Technology of China

DOI:

https://doi.org/10.1609/aaai.v33i01.33012596

Abstract

Reinforcement learning methods have been used for learning dialogue policies. However, learning an effective dialogue policy frequently requires prohibitively many conversations. This is partly because of the sparse rewards in dialogues, and the very few successful dialogues in early learning phase. Hindsight experience replay (HER) enables learning from failures, but the vanilla HER is inapplicable to dialogue learning due to the implicit goals. In this work, we develop two complex HER methods providing different tradeoffs between complexity and performance, and, for the first time, enabled HER-based dialogue policy learning. Experiments using a realistic user simulator show that our HER methods perform better than existing experience replay methods (as applied to deep Q-networks) in learning rate.

Downloads

Published

2019-07-17

How to Cite

Lu, K., Zhang, S., & Chen, X. (2019). Goal-Oriented Dialogue Policy Learning from Failures. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 2596-2603. https://doi.org/10.1609/aaai.v33i01.33012596

Issue

Section

AAAI Technical Track: Humans and AI