Improving Out-of-Scope Detection in Intent Classification by Using Embeddings of the Word Graph Space of the Classes
This paper explores how intent classification can be improved by representing the class labels not as a discrete set of symbols but as a space where the word graphs associated to each class are mapped using typical graph embedding techniques. The approach, inspired by a previous algorithm used for an inverse dictionary task, allows the classification algorithm to take in account inter-class similarities provided by the repeated occurrence of some words in the training examples of the different classes. The classification is carried out by mapping text embeddings to the word graph embeddings of the classes. Focusing solely on improving the representation of the class label set, we show in experiments conducted in both private and public intent classification datasets, that better detection of out-of-scope examples (OOS) is achieved and, as a consequence, that the overall accuracy of intent classification is also improved. In particular, using the recently-released \textit{Larson dataset}, an error of about 9.9{\%} has been achieved for OOS detection, beating the previous state-of-the-art result by more than 31 percentage points.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral ClassificationGraph Embeddingintent-classificationIntent ClassificationSimilar Papers 제목 키워드 기반
Class Embeddings for Improved Out-of-Scope Detection in Intent Classification
This work presents a deep investigation of {\emph class embeddings} for intent classification, which had only been timidly addressed before. We show that there are advantages in representing classes as dense vectors inst…
intent-classificationIntent ClassificationSentenceSentence EmbeddingsA new approach for fine-tuning sentence transformers for intent classification and out-of-scope detection tasks
In virtual assistant (VA) systems it is important to reject or redirect user queries that fall outside the scope of the system. One of the most accurate approaches for out-of-scope (OOS) rejection is to combine it with t…
Classificationintent-classificationIntent ClassificationSentence+2Are Pretrained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection
Pre-trained Transformer-based models were reported to be robust in intent classification. In this work, we first point out the importance of in-domain out-of-scope detection in few-shot intent recognition tasks and then …
intent-classificationIntent ClassificationIntent DetectionIntent RecognitionA Multi-cluster Boundary Learning Method for Out-of-Scope Intent Detection via MiniLM Embedding
Intent detection is a critical task that bridges human intents and system actions in human-machine interaction systems. However, there still exist challenges for detecting out-of-scope (OOS) intents. (i) The traditional …
Multi-class ClassificationIntent DetectionImproved Out-of-Scope Intent Classification with Dual Encoding and Threshold-based Re-Classification
Detecting out-of-scope user utterances is essential for task-oriented dialogues and intent classification. Current methodologies face difficulties with the unpredictable distribution of outliers and often rely on assumpt…
ClassificationDenoisingintent-classificationIntent Classification+1