DeCoOp: Robust Prompt Tuning with Out-of-Distribution Detection
Vision-language models (VLMs), such as CLIP, have demonstrated impressive zero-shot capabilities for various downstream tasks. Their performance can be further enhanced through few-shot prompt tuning methods. However, current studies evaluate the performance of learned prompts separately on base and new classes. This evaluation lacks practicality for real-world applications since downstream tasks cannot determine whether the data belongs to base or new classes in advance. In this paper, we explore a problem setting called Open-world Prompt Tuning (OPT), which involves tuning prompts on base classes and evaluating on a combination of base and new classes. By introducing Decomposed Prompt Tuning framework (DePT), we theoretically demonstrate that OPT can be solved by incorporating out-of-distribution detection into prompt tuning, thereby enhancing the base-to-new discriminability. Based on DePT, we present a novel prompt tuning approach, namely, Decomposed Context Optimization (DeCoOp), which introduces new-class detectors and sub-classifiers to further enhance the base-class and new-class discriminability. Experimental results on 11 benchmark datasets validate the effectiveness of DePT and demonstrate that DeCoOp outperforms current state-of-the-art methods, providing a significant 2% average accuracy improvement.
Code (1)
Tasks
Out-of-Distribution DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Towards Few-shot Out-of-Distribution Detection
Out-of-distribution (OOD) detection is critical for ensuring the reliability of open-world intelligent systems. Despite the notable advancements in existing OOD detection methodologies, our study identifies a significant…
General KnowledgeOut-of-Distribution DetectionOut of Distribution (OOD) DetectionVisual Prompt TuningHow Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models?
Recent large vision-language models such as CLIP have shown remarkable out-of-distribution (OOD) detection and generalization performance. However, their zero-shot in-distribution (ID) accuracy is often limited for downs…
Out-of-Distribution DetectionOut of Distribution (OOD) DetectionPrompt LearningLAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models
Out-of-distribution (OOD) detection is crucial for model reliability, as it identifies samples from unknown classes and reduces errors due to unexpected inputs. Vision-Language Models (VLMs) such as CLIP are emerging as …
Image GenerationOut of Distribution (OOD) DetectionPrompt EngineeringPrompt LearningEnhancing Outlier Knowledge for Few-Shot Out-of-Distribution Detection with Extensible Local Prompts
Out-of-Distribution (OOD) detection, aiming to distinguish outliers from known categories, has gained prominence in practical scenarios. Recently, the advent of vision-language models (VLM) has heightened interest in enh…
Out-of-Distribution DetectionOut of Distribution (OOD) DetectionSelf-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detection
Out-of-distribution (OOD) detection is crucial for deploying reliable machine learning models in open-world applications. Recent advances in CLIP-based OOD detection have shown promising results via regularizing prompt t…
Out-of-Distribution DetectionOut of Distribution (OOD) Detection