SDA: Improving Text Generation with Self Data Augmentation
Data augmentation has been widely used to improve deep neural networks in many research fields, such as computer vision. However, less work has been done in the context of text, partially due to its discrete nature and the complexity of natural languages. In this paper, we propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation. Unlike most existing sentence-level augmentation strategies, which are only applied to specific models, our method is more general and could be easily adapted to any MLE-based training procedure. In addition, our framework allows task-specific evaluation metrics to be designed to flexibly control the generated sentences, for example, in terms of controlling vocabulary usage and avoiding nontrivial repetitions. Extensive experimental results demonstrate the superiority of our method on two synthetic and several standard real datasets, significantly improving related baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationImitation LearningSentenceText GenerationSimilar Papers 제목 키워드 기반
Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation
In task-oriented dialogue systems, response generation from meaning representations (MRs) often suffers from limited training examples, due to the high cost of annotating MR-to-Text pairs. Previous works on self-training…
Dialogue GenerationLanguage ModellingResponse GenerationTask-Oriented Dialogue Systems+1Cropping outperforms dropout as an augmentation strategy for self-supervised training of text embeddings
Text embeddings, i.e. vector representations of entire texts, play an important role in many NLP applications, such as retrieval-augmented generation, clustering, or visualizing collections of texts for data exploration.…
SAS: Self-Augmentation Strategy for Language Model Pre-training
The core of self-supervised learning for pre-training language models includes pre-training task design as well as appropriate data augmentation. Most data augmentations in language model pre-training are context-indepen…
Data AugmentationLanguage ModelingLanguage ModellingMasked Language Modeling+1Improving Text-to-Image Generation with Intrinsic Self-Confidence Rewards
Text-to-image generation powers content creation across design, media, and data augmentation. Post-training of text-to-image generative models is a promising path to improve human preference alignment, factuality, and ae…
Text-to-Image GenerationReinforcement LearningData AugmentationSTA: Self-controlled Text Augmentation for Improving Text Classifications
Despite recent advancements in Machine Learning, many tasks still involve working in low-data regimes which can make solving natural language problems difficult. Recently, a number of text augmentation techniques have em…
BenchmarkingText Augmentation