Generative Adversarial Networks for Annotated Data Augmentation in Data Sparse NLU
Data sparsity is one of the key challenges associated with model development in Natural Language Understanding (NLU) for conversational agents. The challenge is made more complex by the demand for high quality annotated utterances commonly required for supervised learning, usually resulting in weeks of manual labor and high cost. In this paper, we present our results on boosting NLU model performance through training data augmentation using a sequential generative adversarial network (GAN). We explore data generation in the context of two tasks, the bootstrapping of a new language and the handling of low resource features. For both tasks we explore three sequential GAN architectures, one with a token-level reward function, another with our own implementation of a token-level Monte Carlo rollout reward, and a third with sentence-level reward. We evaluate the performance of these feedback models across several sampling methodologies and compare our results to upsampling the original data to the same scale. We further improve the GAN model performance through the transfer learning of the pretrained embeddings. Our experiments reveal synthetic data generated using the sequential generative adversarial network provides significant performance boosts across multiple metrics and can be a major benefit to the NLU tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationGenerative Adversarial NetworkNatural Language UnderstandingSentenceTransfer LearningSimilar Papers 제목 키워드 기반
Adversarial Pulmonary Pathology Translation for Pairwise Chest X-ray Data Augmentation
Recent works show that Generative Adversarial Networks (GANs) can be successfully applied to chest X-ray data augmentation for lung disease recognition. However, the implausible and distorted pathology features generated…
Data AugmentationImage-to-Image TranslationTranslationLearning Data Augmentation for Brain Tumor Segmentation with Coarse-to-Fine Generative Adversarial Networks
There is a common belief that the successful training of deep neural networks requires many annotated training samples, which are often expensive and difficult to obtain especially in the biomedical imaging field. While …
Brain Tumor SegmentationData AugmentationTumor SegmentationTransformer Networks for Data Augmentation of Human Physical Activity Recognition
Data augmentation is a widely used technique in classification to increase data used in training. It improves generalization and reduces amount of annotated human activity data needed for training which reduces labour an…
Activity RecognitionData AugmentationHuman Activity RecognitionTime Series+1A Bayesian Data Augmentation Approach for Learning Deep Models
Data augmentation is an essential part of the training process applied to deep learning models. The motivation is that a robust training process for deep learning models depends on large annotated datasets, which are exp…
Data AugmentationGeneral ClassificationGenerative Adversarial NetworkGenerative Adversarial Networks for Realistic Synthesis of Hyperspectral Samples
This work addresses the scarcity of annotated hyperspectral data required to train deep neural networks. Especially, we investigate generative adversarial networks and their application to the synthesis of consistent lab…
Data Augmentation