Inverse is Better! Fast and Accurate Prompt for Slot Tagging
Prompting methods recently achieve impressive success in few-shot learning. These methods embed input samples with prompt sentence pieces and decode label-related tokens to map samples to the label. However, such a paradigm is very inefficient for the task of slot tagging. Because the slot tagging samples are multiple consecutive words in a sentence, the prompting methods have to enumerate all n-grams token span to find all the possible slots, which greatly slows down the prediction. To tackle this, we introduce an inverse paradigm for prompting. Different from the classic prompts map tokens to labels, we reversely predict slot values given slot types. Such inverse prompting only requires a one-turn prediction for each slot type and greatly speeds up the prediction. Besides, we propose a novel Iterative Prediction Strategy, from which the model learns to refine predictions by considering the relations between different slot types. We find, somewhat surprisingly, the proposed method not only predicts faster, but also significantly improves the effect (improve over 6.1 F1-scores on 10-shot setting) and achieves new state-of-the-art performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningPredictionSentenceSimilar Papers 제목 키워드 기반
Inverse is Better! Fast and Accurate Prompt for Few-shot Slot Tagging
Prompting methods recently achieve impressive success in few-shot learning. These methods modify input samples with prompt sentence pieces, and decode label tokens to map samples to corresponding labels. However, such a …
Few-Shot LearningPredictionSentenceGenerative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Prompting
Zero-shot cross-domain slot filling aims to transfer knowledge from the labeled source domain to the unlabeled target domain. Existing models either encode slot descriptions and examples or design handcrafted question te…
Prompt Learningslot-fillingSlot FillingSLOT: Sample-specific Language Model Optimization at Test-time
We propose SLOT (Sample-specific Language Model Optimization at Test-time), a novel and parameter-efficient test-time inference approach that enhances a language model's ability to more accurately respond to individual p…
GSM8KLanguage ModelingLanguage ModellingModel OptimizationGL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling
Multi-intent SLU can handle multiple intents in an utterance, which has attracted increasing attention. However, the state-of-the-art joint models heavily rely on autoregressive approaches, resulting in two issues: slow …
Intent DetectionSemantic Frame Parsingslot-fillingSlot FillingSlotGCG: Exploiting the Positional Vulnerability in LLMs for Jailbreak Attacks
As large language models (LLMs) are widely deployed, identifying their vulnerability through jailbreak attacks becomes increasingly critical. Optimization-based attacks like Greedy Coordinate Gradient (GCG) have focused …