Complementary Auxiliary Classifiers for Label-Conditional Text Generation

  • Yuan Li Duke University
  • Chunyuan Li Microsoft Research
  • Yizhe Zhang Microsoft Research
  • Xiujun Li Microsoft Research
  • Guoqing Zheng Microsoft Research
  • Lawrence Carin Duke University
  • Jianfeng Gao Microsoft Research

Abstract

Learning to generate text with a given label is a challenging task because natural language sentences are highly variable and ambiguous. It renders difficulties in trade-off between sentence quality and label fidelity. In this paper, we present CARA to alleviate the issue, where two auxiliary classifiers work simultaneously to ensure that (1) the encoder learns disentangled features and (2) the generator produces label-related sentences. Two practical techniques are further proposed to improve the performance, including annealing the learning signal from the auxiliary classifier, and enhancing the encoder with pre-trained language models. To establish a comprehensive benchmark fostering future research, we consider a suite of four datasets, and systematically reproduce three representative methods. CARA shows consistent improvement over the previous methods on the task of label-conditional text generation, and achieves state-of-the-art on the task of attribute transfer.

Published
2020-04-03
Section
AAAI Technical Track: Natural Language Processing