CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection
In this paper, we formulate a more realistic and difficult problem setup for the intent detection task in natural language understanding, namely Generalized Few-Shot Intent Detection (GFSID). GFSID aims to discriminate a joint label space consisting of both existing intents which have enough labeled data and novel intents which only have a few examples for each class. To approach this problem, we propose a novel model, Conditional Text Generation with BERT (CG-BERT). CG-BERT effectively leverages a large pre-trained language model to generate text conditioned on the intent label. By modeling the utterance distribution with variational inference, CG-BERT can generate diverse utterances for the novel intents even with only a few utterances available. Experimental results show that CG-BERT achieves state-of-the-art performance on the GFSID task with 1-shot and 5-shot settings on two real-world datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Conditional Text GenerationIntent DetectionLanguage ModelingLanguage ModellingNatural Language UnderstandingText GenerationVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Distilling Knowledge Learned in BERT for Text Generation
Large-scale pre-trained language model such as BERT has achieved great success in language understanding tasks. However, it remains an open question how to utilize BERT for language generation. In this paper, we present …
Language ModelingLanguage ModellingMachine TranslationMasked Language Modeling+4Conditional BERT Contextual Augmentation
We propose a novel data augmentation method for labeled sentences called conditional BERT contextual augmentation. Data augmentation methods are often applied to prevent overfitting and improve generalization of deep neu…
Data AugmentationLanguage ModelingLanguage ModellingText ClassificationSelf-Training Vision Language BERTs with a Unified Conditional Model
Natural language BERTs are trained with language corpus in a self-supervised manner. Unlike natural language BERTs, vision language BERTs need paired data to train, which restricts the scale of VL-BERT pretraining. We pr…
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
There have been various types of pretraining architectures including autoencoding models (e.g., BERT), autoregressive models (e.g., GPT), and encoder-decoder models (e.g., T5). However, none of the pretraining frameworks…
Abstractive Text SummarizationClassificationDecoderDocument Summarization+4GLM: General Language Model Pretraining with Autoregressive Blank Infilling
There have been various types of pretraining architectures including autoencoding models (e.g., BERT), autoregressive models (e.g., GPT), and encoder-decoder models (e.g., T5). However, none of the pretraining frameworks…
DecoderLanguage ModelingLanguage ModellingNatural Language Understanding