Scalable Prompt Generation for Semi-supervised Learning with Language Models
Prompt-based learning methods in semi-supervised learning (SSL) settings have been shown to be effective on multiple natural language understanding (NLU) datasets and tasks in the literature. However, manually designing multiple prompts and verbalizers requires domain knowledge and human effort, making it difficult and expensive to scale across different datasets. In this paper, we propose two methods to automatically design multiple prompts and integrate automatic verbalizer in SSL settings without sacrificing performance. The first method uses various demonstration examples with learnable continuous prompt tokens to create diverse prompt models. The second method uses a varying number of soft prompt tokens to encourage language models to learn different prompts. For the verbalizer, we use the prototypical verbalizer to replace the manual one. In summary, we obtained the best average accuracy of 73.2% (a relative improvement of 2.52% over even the previous state-of-the-art SSL method with manual prompts and verbalizers) in different few-shot learning settings.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningNatural Language UnderstandingSimilar Papers 제목 키워드 기반
A Semi-Supervised Framework for Breast Ultrasound Segmentation with Training-Free Pseudo-Label Generation and Label Refinement
Semi-supervised learning (SSL) has emerged as a promising paradigm for breast ultrasound (BUS) image segmentation, but it often suffers from unstable pseudo labels under extremely limited annotations, leading to inaccura…
Semi-supervised Medical Image SegmentationContrastive LearningCPS4: Class Prompt driven Semi-Supervised Spine Segmentation with Class-specific Consistency Constraint
Vision Language Model (VLM) has great potential to enhance the quality of pseudo labels in semi-supervised spine segmentation by leveraging textual class prompts to generate segmentation map, but no one has studied it ye…
Pre-trained Prompt-driven Semi-supervised Local Community Detection
Semi-supervised local community detection aims to leverage known communities to detect the community containing a given node. Although existing semi-supervised local community detection studies yield promising results, t…
Community DetectionCommunity SearchLocal Community DetectionMultistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation
We study semi-supervised sequence generation tasks, where the few labeled examples are too scarce to finetune a model, and meanwhile, few-shot prompted large language models (LLMs) exhibit room for improvement. In this p…
Constituency ParsingKnowledge DistillationLanguage ModelingLanguage Modelling+2SelfPrompt: Confidence-Aware Semi-Supervised Tuning for Robust Vision-Language Model Adaptation
We present SelfPrompt, a novel prompt-tuning approach for vision-language models (VLMs) in a semi-supervised learning setup. Existing methods for tuning VLMs in semi-supervised setups struggle with the negative impact of…
Language ModelingLanguage ModellingPseudo LabelWeakly-supervised Learning