CQARE: Contrastive Question-Answering for Few-shot Relation Extraction with Prompt Tuning
Prompt tuning with pre-trained language models (PLM) has exhibited outstanding performance by closing the gap between pre-training tasks and various downstream applications, without the need for uninitialized parameters to be introduced. However, prompt tuning requires vast amounts of prompt engineering and predefined label word mapping, which obstructs its implements in practice. Besides, the ample label space makes prompt tuning more arduous and challenging when it comes to relation extraction (RE). To tackle these issues, we propose a Contrastive Question-Answering method with prompt tuning for few-shot RE (CQARE). CQARE carries out a RE task-specific pre-training with four entity-relation-aware pre-training objects, including a prompt pre-training to automatically generate continuous prompts. The proposed pre-training can provide more robust initialization with prompt tuning while maintaining semantic consistency with the proposed PLM. Furthermore, CQARE can effectively avoid label words mapping by reformulating RE as contrastive question answering. The results indicate CQARE raising averaged accuracy of 5.11\% on a cross-domain few-shot dataset, demonstrating that robust initialization is crucial for prompt tuning and effective contrastive question answering.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Domain Few-ShotPrompt EngineeringQuestion AnsweringRelationRelation ExtractionSimilar Papers 제목 키워드 기반
Momentum Contrastive Pre-training for Question Answering
Existing pre-training methods for extractive Question Answering (QA) generate cloze-like queries different from natural questions in syntax structure, which could overfit pre-trained models to simple keyword matching. In…
BenchmarkingContrastive LearningExtractive Question-AnsweringNatural Questions+1Multi-hop Commonsense Knowledge Injection Framework for Zero-Shot Commonsense Question Answering
Commonsense question answering (QA) research requires machines to answer questions based on commonsense knowledge. However, this research requires expensive labor costs to annotate data as the basis of research, and mode…
Contrastive LearningKnowledge GraphsQuestion AnsweringMICO: A Multi-alternative Contrastive Learning Framework for Commonsense Knowledge Representation
Commonsense reasoning tasks such as commonsense knowledge graph completion and commonsense question answering require powerful representation learning. In this paper, we propose to learn commonsense knowledge representat…
Contrastive LearningKnowledge Graph CompletionKnowledge GraphsQuestion Answering+2QAID: Question Answering Inspired Few-shot Intent Detection
Intent detection with semantically similar fine-grained intents is a challenging task. To address it, we reformulate intent detection as a question-answering retrieval task by treating utterances and intent names as ques…
Intent DetectionQuestion AnsweringRetrievalKECP: Knowledge Enhanced Contrastive Prompting for Few-shot Extractive Question Answering
Extractive Question Answering (EQA) is one of the most important tasks in Machine Reading Comprehension (MRC), which can be solved by fine-tuning the span selecting heads of Pre-trained Language Models (PLMs). However, m…
Contrastive LearningExtractive Question-AnsweringFew-Shot LearningLanguage Modeling+5