DAWSON: Data Augmentation using Weak Supervision On Natural Language
We propose a novel data augmentation model for text, using all available data through weak supervision. To improve generalization, recent work in the field uses BERT and masked language modeling to conditionally augment data. These models all involve a small, high-quality labeled dataset, but omit the abundance of unlabeled data, which is likely to be present if one considers a model in the first place. Weak supervision methods, such as Snorkel, make use of the vastness of unlabeled data, but largely omit the available ground truth labels. We combine data augmentation and weak supervision techniques into a holistic method, consisting of 4 training phases and 2 inference phases, to efficiently train an end-to-end model when only a small amount of annotated data is available. We outperform the benchmark (Kumar et al.,2020) for the SST-2 task by 1.5, QQP task by 4.4, and QNLI task by 3.0 absolute accuracy points, and show that data augmentation is also effective for natural language understanding tasks, such as QQP and QNLI.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationLanguage ModelingLanguage ModellingMasked Language ModelingNatural Language UnderstandingQNLIQQPSST-2Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DAWSON: A Domain Adaptive Few Shot Generation Framework
Training a Generative Adversarial Networks (GAN) for a new domain from scratch requires an enormous amount of training data and days of training time. To this end, we propose DAWSON, a Domain Adaptive FewShot Generation …
Meta-LearningMusic GenerationImproving the performance of weak supervision searches using data augmentation
Weak supervision combines the advantages of training on real data with the ability to exploit signal properties. However, training a neural network using weak supervision often requires an excessive amount of signal data…
Data AugmentationDiversityGenerative Modeling Helps Weak Supervision (and Vice Versa)
Many promising applications of supervised machine learning face hurdles in the acquisition of labeled data in sufficient quantity and quality, creating an expensive bottleneck. To overcome such limitations, techniques th…
Data Augmentationimage-classificationImage ClassificationHierarchical Supervision and Shuffle Data Augmentation for 3D Semi-Supervised Object Detection
State-of-the-art 3D object detectors are usually trained on large-scale datasets with high-quality 3D annotations. However, such 3D annotations are often expensive and time-consuming, which may not be practical for real …
Data Augmentationobject-detectionObject DetectionSemi-Supervised Object DetectionWALNUT: A Benchmark on Semi-weakly Supervised Learning for Natural Language Understanding
Building machine learning models for natural language understanding (NLU) tasks relies heavily on labeled data. Weak supervision has been proven valuable when large amount of labeled data is unavailable or expensive to o…
Natural Language UnderstandingWeakly-supervised Learning