Discriminative Language Model as Semantic Consistency Scorer for Prompt-based Few-Shot Text Classification
This paper proposes a novel prompt-based finetuning method (called DLM-SCS) for few-shot text classification by utilizing the discriminative language model ELECTRA that is pretrained to distinguish whether a token is original or generated. The underlying idea is that the prompt instantiated with the true label should have higher semantic consistency score than other prompts with false labels. Since a prompt usually consists of several components (or parts), its semantic consistency can be decomposed accordingly. The semantic consistency of each component is then computed by making use of the pretrained ELECTRA model, without introducing extra parameters. Extensive experiments have shown that our model outperforms several state-of-the-art prompt-based few-shot methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot Text ClassificationLanguage ModelingLanguage Modellingtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Equal Ranking Quality, Different Decisions: Training Order-Consistent LLM Scorers
Rerankers, reward models and multi-document QA scorers score candidate documents or responses in one LLM prompt, so each score depends on their order. Such scorers are selected on ranking quality, but their scores determ…
LLM as a Scorer: The Impact of Output Order on Dialogue Evaluation
This research investigates the effect of prompt design on dialogue evaluation using large language models (LLMs). While LLMs are increasingly used for scoring various inputs, creating effective prompts for dialogue evalu…
Dialogue EvaluationSensitivityDetecting Off-topic Responses to Visual Prompts
Automated methods for essay scoring have made great progress in recent years, achieving accuracies very close to human annotators. However, a known weakness of such automated scorers is not taking into account the semant…
Training-Free Multimodal Deepfake Detection via Graph Reasoning
Multimodal deepfake detection (MDD) aims to uncover manipulations across visual, textual, and auditory modalities, thereby reinforcing the reliability of modern information systems. Although large vision-language models …
Multimodal ReasoningDeepFake DetectionMulti-Stage Pre-training for Automated Chinese Essay Scoring
This paper proposes a pre-training based automated Chinese essay scoring method. The method involves three components: weakly supervised pre-training, supervised cross- prompt fine-tuning and supervised target- prompt fi…
Domain Adaptation