Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP
In an out-of-distribution (OOD) detection problem, samples of known classes(also called in-distribution classes) are used to train a special classifier. In testing, the classifier can (1) classify the test samples of known classes to their respective classes and also (2) detect samples that do not belong to any of the known classes (i.e., they belong to some unknown or OOD classes). This paper studies the problem of zero-shot out-of-distribution(OOD) detection, which still performs the same two tasks in testing but has no training except using the given known class names. This paper proposes a novel yet simple method (called ZOC) to solve the problem. ZOC builds on top of the recent advances in zero-shot classification through multi-modal representation learning. It first extends the pre-trained language-vision model CLIP by training a text-based image description generator on top of CLIP. In testing, it uses the extended model to generate candidate unknown class names for each test sample and computes a confidence score based on both the known class names and candidate unknown class names for zero-shot OOD detection. Experimental results on 5 benchmark datasets for OOD detection demonstrate that ZOC outperforms the baselines by a large margin.
Code (2)
Tasks
Image DescriptionOut-of-Distribution DetectionOut of Distribution (OOD) DetectionRepresentation Learningzero-shot-classificationZero-Shot LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
COOkeD: Ensemble-based OOD detection in the era of zero-shot CLIP
Out-of-distribution (OOD) detection is an important building block in trustworthy image recognition systems as unknown classes may arise at test-time. OOD detection methods typically revolve around a single classifier, l…
CLIP4HOI: Towards Adapting CLIP for Practical Zero-Shot HOI Detection
Zero-shot Human-Object Interaction (HOI) detection aims to identify both seen and unseen HOI categories. A strong zero-shot HOI detector is supposed to be not only capable of discriminating novel interactions but also ro…
VisTa: Visual-contextual and Text-augmented Zero-shot Object-level OOD Detection
As object detectors are increasingly deployed as black-box cloud services or pre-trained models with restricted access to the original training data, the challenge of zero-shot object-level out-of-distribution (OOD) dete…
ObjectOut of Distribution (OOD) DetectionCLIPScope: Enhancing Zero-Shot OOD Detection with Bayesian Scoring
Detection of out-of-distribution (OOD) samples is crucial for safe real-world deployment of machine learning models. Recent advances in vision language foundation models have made them capable of detecting OOD samples wi…
GP-Adapter: Gaussian Process CLIP-Adapter for Few-Shot Out-of-Distribution Detection
We propose GP-Adapter, a training-free framework that augments CLIP (Contrastive Language-Image Pre-training) with Gaussian Process (GP) uncertainty modeling for few-shot classification and out-of-distribution (OOD) dete…
Out-of-Distribution Detection