Neural Data Augmentation via Example Extrapolation
In many applications of machine learning, certain categories of examples may be underrepresented in the training data, causing systems to underperform on such "few-shot" cases at test time. A common remedy is to perform data augmentation, such as by duplicating underrepresented examples, or heuristically synthesizing new examples. But these remedies often fail to cover the full diversity and complexity of real examples. We propose a data augmentation approach that performs neural Example Extrapolation (Ex2). Given a handful of exemplars sampled from some distribution, Ex2 synthesizes new examples that also belong to the same distribution. The Ex2 model is learned by simulating the example generation procedure on data-rich slices of the data, and it is applied to underrepresented, few-shot slices. We apply Ex2 to a range of language understanding tasks and significantly improve over state-of-the-art methods on multiple few-shot learning benchmarks, including for relation extraction (FewRel) and intent classification + slot filling (SNIPS).
Code (1)
Tasks
Data AugmentationDiversityFew-Shot Learningintent-classificationIntent ClassificationRelation Extractionslot-fillingSlot FillingSimilar Papers 제목 키워드 기반
Good-Enough Example Extrapolation
This paper asks whether extrapolating the hidden space distribution of text examples from one class onto another is a valid inductive bias for data augmentation. To operationalize this question, I propose a simple data a…
Data AugmentationInductive Biastext-classificationText Classification+1Teaching by Failure: Counter-Example-Driven Curricula for Transformer Self-Improvement
Transformer models often exhibit brittle extrapolation, failing on inputs that are longer or structurally more complex than those seen during training. We introduce Counter-Example-Driven Curricula (CEDC), an automated f…
Text ClassificationData AugmentationReprint: a randomized extrapolation based on principal components for data augmentation
Data scarcity and data imbalance have attracted a lot of attention in many fields. Data augmentation, explored as an effective approach to tackle them, can improve the robustness and efficiency of classification models b…
Data Augmentationtext-classificationText ClassificationFactorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation
Expanding an existing tourist photo from a partially captured scene to a full scene is one of the desired experiences for photography applications. Although photo extrapolation has been well studied, it is much more chal…
Learning to Recombine and Resample Data for Compositional Generalization
Flexible neural sequence models outperform grammar- and automaton-based counterparts on a variety of tasks. However, neural models perform poorly in settings requiring compositional generalization beyond the training dat…
Data AugmentationInstruction FollowingMorphological Analysis