When Low Resource NLP Meets Unsupervised Language Model: Meta-Pretraining then Meta-Learning for Few-Shot Text Classification (Student Abstract)

Authors

  • Shumin Deng Zhejiang University
  • Ningyu Zhang Alibaba-Zhejiang University
  • Zhanlin Sun Alibaba-Zhejiang University
  • Jiaoyan Chen Oxford University
  • Huajun Chen Zhejiang University

DOI:

https://doi.org/10.1609/aaai.v34i10.7158

Abstract

Text classification tends to be difficult when data are deficient or when it is required to adapt to unseen classes. In such challenging scenarios, recent studies have often used meta-learning to simulate the few-shot task, thus negating implicit common linguistic features across tasks. This paper addresses such problems using meta-learning and unsupervised language models. Our approach is based on the insight that having a good generalization from a few examples relies on both a generic model initialization and an effective strategy for adapting this model to newly arising tasks. We show that our approach is not only simple but also produces a state-of-the-art performance on a well-studied sentiment classification dataset. It can thus be further suggested that pretraining could be a promising solution for few-shot learning of many other NLP tasks. The code and the dataset to replicate the experiments are made available at https://github.com/zxlzr/FewShotNLP.

Downloads

Published

2020-04-03

How to Cite

Deng, S., Zhang, N., Sun, Z., Chen, J., & Chen, H. (2020). When Low Resource NLP Meets Unsupervised Language Model: Meta-Pretraining then Meta-Learning for Few-Shot Text Classification (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 34(10), 13773-13774. https://doi.org/10.1609/aaai.v34i10.7158

Issue

Section

Student Abstract Track