Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble
Out-of-distribution (OOD) detection aims to discern outliers from the intended data distribution, which is crucial to maintaining high reliability and a good user experience. Most recent studies in OOD detection utilize the information from a single representation that resides in the penultimate layer to determine whether the input is anomalous or not. Although such a method is straightforward, the potential of diverse information in the intermediate layers is overlooked. In this paper, we propose a novel framework based on contrastive learning that encourages intermediate features to learn layer-specialized representations and assembles them implicitly into a single representation to absorb rich information in the pre-trained language model. Extensive experiments in various intent classification and OOD datasets demonstrate that our approach is significantly more effective than other works.
Code (1)
Tasks
Contrastive Learningintent-classificationIntent ClassificationLanguage ModelingLanguage ModellingNatural Language UnderstandingOut-of-Distribution DetectionOut of Distribution (OOD) DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
How Good Are LLMs at Out-of-Distribution Detection?
Out-of-distribution (OOD) detection plays a vital role in enhancing the reliability of machine learning (ML) models. The emergence of large language models (LLMs) has catalyzed a paradigm shift within the ML community, s…
Out-of-Distribution DetectionOut of Distribution (OOD) DetectionEnhancing Cross-lingual Natural Language Inference by Prompt-learning from Cross-lingual Templates
Cross-lingual natural language inference (XNLI) is a fundamental task in cross-lingual natural language understanding. Recently this task is commonly addressed by pre-trained cross-lingual language models. Existing metho…
Cross-Lingual Natural Language InferenceCross-Lingual TransferLanguage ModelingLanguage Modelling+4Enhancing Slot Tagging with Intent Features for Task Oriented Natural Language Understanding using BERT
Recent joint intent detection and slot tagging models have seen improved performance when compared to individual models. In many real-world datasets, the slot labels and values have a strong correlation with their intent…
Intent DetectionNatural Language UnderstandingSemantic-Guided Natural Language and Visual Fusion for Cross-Modal Interaction Based on Tiny Object Detection
This paper introduces a cutting-edge approach to cross-modal interaction for tiny object detection by combining semantic-guided natural language processing with advanced visual recognition backbones. The proposed method …
Natural Language UnderstandingObject DetectionVision-Language Integration for Zero-Shot Scene Understanding in Real-World Environments
Zero-shot scene understanding in real-world settings presents major challenges due to the complexity and variability of natural scenes, where models must recognize new objects, actions, and contexts without prior labeled…
Scene UnderstandingActivity DetectionObject Recognition