DuNST: Dual Noisy Self Training for Semi-Supervised Controllable Text Generation
Self-training (ST) has prospered again in language understanding by augmenting the fine-tuning of pre-trained language models when labeled data is insufficient. However, it remains challenging to incorporate ST into attribute-controllable language generation. Augmented by only self-generated pseudo text, generation models over-emphasize exploitation of the previously learned space, suffering from a constrained generalization boundary. We revisit ST and propose a novel method, DuNST to alleviate this problem. DuNST jointly models text generation and classification with a shared Variational AutoEncoder and corrupts the generated pseudo text by two kinds of flexible noise to disturb the space. In this way, our model could construct and utilize both pseudo text from given labels and pseudo labels from available unlabeled text, which are gradually refined during the ST process. We theoretically demonstrate that DuNST can be regarded as enhancing exploration towards the potential real text space, providing a guarantee of improved performance. Experiments on three controllable generation tasks show that DuNST could significantly boost control accuracy while maintaining comparable generation fluency and diversity against several strong baselines.
Code (1)
Tasks
AttributeDiversityText GenerationSimilar Papers 제목 키워드 기반
Self-semi-supervised Learning to Learn from NoisyLabeled Data
The remarkable success of today's deep neural networks highly depends on a massive number of correctly labeled data. However, it is rather costly to obtain high-quality human-labeled data, leading to the active research …
Self-training with noisy student model and semi-supervised loss function for dcase 2021 challenge task 4
This report proposes a polyphonic sound event detection (SED) method for the DCASE 2021 Challenge Task 4. The proposed SED model consists of two stages: a mean-teacher model for providing target labels regarding weakly l…
Event DetectionSound Event DetectionSemi-supervised Relation Extraction via Incremental Meta Self-Training
To alleviate human efforts from obtaining large-scale annotations, Semi-Supervised Relation Extraction methods aim to leverage unlabeled data in addition to learning from limited samples. Existing self-training methods s…
Meta-LearningPseudo LabelRelationRelation Classification+1Dual Invariance Self-training for Reliable Semi-supervised Surgical Phase Recognition
Accurate surgical phase recognition is crucial for advancing computer-assisted interventions, yet the scarcity of labeled data hinders training reliable deep learning models. Semi-supervised learning (SSL), particularly …
Pseudo LabelSurgical phase recognitionSELF: Learning to Filter Noisy Labels with Self-Ensembling
Deep neural networks (DNNs) have been shown to over-fit a dataset when being trained with noisy labels for a long enough time. To overcome this problem, we present a simple and effective method self-ensemble label filter…
image-classificationImage ClassificationSelf-Learning