GenAug: Data Augmentation for Finetuning Text Generators
In this paper, we investigate data augmentation for text generation, which we call GenAug. Text generation and language modeling are important tasks within natural language processing, and are especially challenging for low-data regimes. We propose and evaluate various augmentation methods, including some that incorporate external knowledge, for finetuning GPT-2 on a subset of Yelp Reviews. We also examine the relationship between the amount of augmentation and the quality of the generated text. We utilize several metrics that evaluate important aspects of the generated text including its diversity and fluency. Our experiments demonstrate that insertion of character-level synthetic noise and keyword replacement with hypernyms are effective augmentation methods, and that the quality of generations improves to a peak at approximately three times the amount of original data.
Code (2)
Tasks
Data AugmentationDiversityLanguage ModelingLanguage ModellingText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Ctrl-GenAug: Controllable Generative Augmentation for Medical Sequence Classification
In the medical field, the limited availability of large-scale datasets and labor-intensive annotation processes hinder the performance of deep models. Diffusion-based generative augmentation approaches present a promisin…
Improving Model Factuality with Fine-grained Critique-based Evaluator
Factuality evaluation aims to detect factual errors produced by language models (LMs) and hence guide the development of more factual models. Towards this goal, we train a factuality evaluator, FenCE, that provides LM ge…
Data AugmentationTailor: Generating and Perturbing Text with Semantic Controls
Making controlled perturbations is essential for various tasks (e.g., data augmentation), but building task-specific generators can be expensive. We introduce Tailor, a task-agnostic generation system that perturbs text …
Data AugmentationNatural Language InferenceStyle TransferFinetuning Is a Surprisingly Effective Domain Adaptation Baseline in Handwriting Recognition
In many machine learning tasks, a large general dataset and a small specialized dataset are available. In such situations, various domain adaptation methods can be used to adapt a general model to the target dataset. We …
Data AugmentationDomain AdaptationHandwriting RecognitionNot Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generators
Neural image classifiers are known to undergo severe performance degradation when exposed to inputs that are sampled from environmental conditions that differ from their training data. Given the recent progress in Text-t…
Data AugmentationDomain GeneralizationImage AugmentationRetrieval