Outlier Detection for Improved Data Quality and Diversity in Dialog Systems
In a corpus of data, outliers are either errors: mistakes in the data that are counterproductive, or are unique: informative samples that improve model robustness. Identifying outliers can lead to better datasets by (1) removing noise in datasets and (2) guiding collection of additional data to fill gaps. However, the problem of detecting both outlier types has received relatively little attention in NLP, particularly for dialog systems. We introduce a simple and effective technique for detecting both erroneous and unique samples in a corpus of short texts using neural sentence embeddings combined with distance-based outlier detection. We also present a novel data collection pipeline built atop our detection technique to automatically and iteratively mine unique data samples while discarding erroneous samples. Experiments show that our outlier detection technique is effective at finding errors while our data collection pipeline yields highly diverse corpora that in turn produce more robust intent classification and slot-filling models.
Code (0)
등록된 구현이 없습니다.
Tasks
Diversityintent-classificationIntent ClassificationIntent Classification and Slot FillingOutlier DetectionSentenceSentence Embeddingsslot-fillingSlot FillingSimilar Papers 제목 키워드 기반
Unsupervised Time Series Outlier Detection with Diversity-Driven Convolutional Ensembles -- Extended Version
With the sweeping digitalization of societal, medical, industrial, and scientific processes, sensing technologies are being deployed that produce increasing volumes of time series data, thus fueling a plethora of new or …
DiversityOutlier DetectionTime SeriesTime Series AnalysisUnsupervised Boosting-based Autoencoder Ensembles for Outlier Detection
Autoencoders, as a dimensionality reduction technique, have been recently applied to outlier detection. However, neural networks are known to be vulnerable to overfitting, and therefore have limited potential in the unsu…
Dimensionality ReductionDiversityOutlier DetectionOut-Of-Distribution Detection with Diversification (Provably)
Out-of-distribution (OOD) detection is crucial for ensuring reliable deployment of machine learning models. Recent advancements focus on utilizing easily accessible auxiliary outliers (e.g., data from the web or other da…
DiversityOut-of-Distribution DetectionOut of Distribution (OOD) DetectionHGOE: Hybrid External and Internal Graph Outlier Exposure for Graph Out-of-Distribution Detection
With the progressive advancements in deep graph learning, out-of-distribution (OOD) detection for graph data has emerged as a critical challenge. While the efficacy of auxiliary datasets in enhancing OOD detection has be…
Outlier Detection using Improved Genetic K-means
The outlier detection problem in some cases is similar to the classification problem. For example, the main concern of clustering-based outlier detection algorithms is to find clusters and outliers, which are often regar…
ClusteringGeneral ClassificationOutlier Detection