Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation
Despite the success of text-to-text pre-trained models in various natural language generation (NLG) tasks, the generation performance is largely restricted by the number of labeled data in downstream tasks, particularly in data-to-text generation tasks. Existing works mostly utilize abundant unlabeled structured data to conduct unsupervised pre-training for task adaption, which fail to model the complex relationship between source structured data and target texts. Thus, we introduce self-training as a better few-shot learner than task-adaptive pre-training, which explicitly captures this relationship via pseudo-labeled data generated by the pre-trained model. To alleviate the side-effect of low-quality pseudo-labeled data during self-training, we propose a novel method called Curriculum-Based Self-Training (CBST) to effectively leverage unlabeled data in a rearranged order determined by the difficulty of text generation. Experimental results show that our method can outperform fine-tuning and task-adaptive pre-training methods, and achieve state-of-the-art performance in the few-shot setting of data-to-text generation.
Code (1)
Tasks
Data-to-Text GenerationText GenerationUnsupervised Pre-trainingSimilar Papers 제목 키워드 기반
Worst-Case-Aware Curriculum Learning for Zero and Few Shot Transfer
Multi-task transfer learning based on pre-trained language encoders achieves state-of-the-art performance across a range of tasks. Standard approaches implicitly assume the tasks, for which we have training data, are equ…
Transfer LearningCDFSL-V: Cross-Domain Few-Shot Learning for Videos
Few-shot video action recognition is an effective approach to recognizing new categories with only a few labeled examples, thereby reducing the challenges associated with collecting and annotating large-scale video datas…
Action RecognitionCross-Domain Few-Shotcross-domain few-shot learningFew-Shot action recognition+4ANCORA: Learning to Question via Manifold-Anchored Self-Play for Verifiable Reasoning
We propose a paradigm shift toward open-ended curriculum self-play: rather than learning to answer on a fixed prompt set, a unified policy learns to question: generating verifiable problems, solving them, and turning ver…
A Self-Adaptive Learning Rate and Curriculum Learning Based Framework for Few-Shot Text Classification
Due to the lack of labeled data in many realistic scenarios, a number of few-shot learning methods for text classification have been proposed, among which the meta learning based ones have recently attracted much attenti…
Few-Shot LearningFew-Shot Text ClassificationMeta-Learningtext-classification+1Training with Growing Sets: A Simple Alternative to Curriculum Learning and Self Paced Learning
Curriculum learning and Self paced learning are popular topics in the machine learning that suggest to put the training samples in order by considering their difficulty levels. Studies in these topics show that starting …