Delving into Out-of-Distribution Detection with Vision-Language Representations
Recognizing out-of-distribution (OOD) samples is critical for machine learning systems deployed in the open world. The vast majority of OOD detection methods are driven by a single modality (e.g., either vision or language), leaving the rich information in multi-modal representations untapped. Inspired by the recent success of vision-language pre-training, this paper enriches the landscape of OOD detection from a single-modal to a multi-modal regime. Particularly, we propose Maximum Concept Matching (MCM), a simple yet effective zero-shot OOD detection method based on aligning visual features with textual concepts. We contribute in-depth analysis and theoretical insights to understand the effectiveness of MCM. Extensive experiments demonstrate that MCM achieves superior performance on a wide variety of real-world tasks. MCM with vision-language features outperforms a common baseline with pure visual features on a hard OOD task with semantically similar classes by 13.1% (AUROC). Code is available at https://github.com/deeplearning-wisc/MCM.
Code (2)
Tasks
Out-of-Distribution DetectionSimilar Papers 제목 키워드 기반
Delving into Out-of-Distribution Detection with Medical Vision-Language Models
Recent advances in medical vision-language models (VLMs) demonstrate impressive performance in image classification tasks, driven by their strong zero-shot generalization capabilities. However, given the high variability…
Benchmarkingimage-classificationImage ClassificationOut-of-Distribution Detection+1OT-DETECTOR: Delving into Optimal Transport for Zero-shot Out-of-Distribution Detection
Out-of-distribution (OOD) detection is crucial for ensuring the reliability and safety of machine learning models in real-world applications. While zero-shot OOD detection, which requires no training on in-distribution (…
Out-of-Distribution DetectionOut of Distribution (OOD) DetectionFA: Forced Prompt Learning of Vision-Language Models for Out-of-Distribution Detection
Pre-trained vision-language models (VLMs) have advanced out-of-distribution (OOD) detection recently. However, existing CLIP-based methods often focus on learning OOD-related knowledge to improve OOD detection, showing l…
Out-of-Distribution DetectionOut of Distribution (OOD) DetectionPrompt LearningSemantic Similarity+1CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection
This paper considers zero-shot Anomaly Detection (AD), performing AD without reference images of the test objects. We propose a framework called CLIP-AD to leverage the zero-shot capabilities of the large vision-language…
Anomaly DetectionLanguage ModelingLanguage Modellingzero-shot anomaly detection+1CLIPSym: Delving into Symmetry Detection with CLIP
Symmetry is one of the most fundamental geometric cues in computer vision, and detecting it has been an ongoing challenge. With the recent advances in vision-language models,~i.e., CLIP, we investigate whether a pre-trai…
Symmetry Detection