Technical Report -- Competition Solution for Prompt Tuning using Pretrained Language Model
Prompt tuning recently becomes a hot-spot in the applications of large pretrained language models on specific downstream tasks. Regarding the Language Model as a Service (LMaaS), black-box tuning using derivative-free optimization (DFO) provides a novel approach to expand the practical scenarios of pretrained models and enrich the researches of few-shot learning. In this report, we present our solution in this competition that is based on the LMaaS scenario. Our solution consists of several modifications to BBTv2, including multiple label words, selection of P0, rolling update strategy, multi-task loss from MLP classifier, and finally using the ensemble method to further improve generalization ability. We also shared some strategies that we tried but didn't use in the final submission for further discussion. In the end we raised a question about the SNLI dataset and the impact on the results, as well as our concerns about the competition.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningLanguage ModelingLanguage ModellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Technical Report: Competition Solution For Modelscope-Sora
This report presents the approach adopted in the Modelscope-Sora challenge, which focuses on fine-tuning data for video generation models. The challenge evaluates participants' ability to analyze, clean, and generate hig…
Text-to-Video GenerationVideo DescriptionVideo GenerationTechnical Report: Competition Solution For BetterMixture
In the era of flourishing large-scale models, the challenge of selecting and optimizing datasets from the vast and complex sea of data, to enhance the performance of large language models within the constraints of limite…
DiversityICME 2022 Few-shot LOGO detection top 9 solution
ICME-2022 few-shot logo detection competition is held in May, 2022. Participants are required to develop a single model to detect logos by handling tiny logo instances, similar brands, and adversarial images at the same …
First Place Solution of 2023 Global Artificial Intelligence Technology Innovation Competition Track 1
In this paper, we present our champion solution to the Global Artificial Intelligence Technology Innovation Competition Track 1: Medical Imaging Diagnosis Report Generation. We select CPT-BASE as our base model for the t…
DenoisingDiagnosticLanguage ModelingLanguage Modelling+2Alibaba International E-commerce Product Search Competition DcuRAGONs Team Technical Report
This report details our methodology and results developed for the Multilingual E-commerce Search Competition. The problem aims to recognize relevance between user queries versus product items in a multilingual context an…