Multi-method Integration with Confidence-based Weighting for Zero-shot Image Classification
This paper introduces a novel framework for zero-shot learning (ZSL), i.e., to recognize new categories that are unseen during training, by using a multi-model and multi-alignment integration method. Specifically, we propose three strategies to enhance the model's performance to handle ZSL: 1) Utilizing the extensive knowledge of ChatGPT and the powerful image generation capabilities of DALL-E to create reference images that can precisely describe unseen categories and classification boundaries, thereby alleviating the information bottleneck issue; 2) Integrating the results of text-image alignment and image-image alignment from CLIP, along with the image-image alignment results from DINO, to achieve more accurate predictions; 3) Introducing an adaptive weighting mechanism based on confidence levels to aggregate the outcomes from different prediction methods. Experimental results on multiple datasets, including CIFAR-10, CIFAR-100, and TinyImageNet, demonstrate that our model can significantly improve classification accuracy compared to single-model approaches, achieving AUROC scores above 96% across all test datasets, and notably surpassing 99% on the CIFAR-10 dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
image-classificationImage ClassificationImage GenerationZero-Shot Image ClassificationZero-Shot LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Adversarial Robustness of Zero-Shot CLIP with Confidence-Aware Weighting
Vision-language models such as CLIP demonstrate impressive zero-shot generalization but remain highly vulnerable to adversarial attacks. Prior adversarial methods treat all samples equally in the loss function, despite t…
Zero-shot GeneralizationLeveraging Prediction Entropy for Automatic Prompt Weighting in Zero-Shot Audio-Language Classification
Audio-language models have recently demonstrated strong zero-shot capabilities by leveraging natural-language supervision to classify audio events without labeled training data. Yet, their performance is highly sensitive…
Audio ClassificationImproving Cross-lingual Text Classification with Zero-shot Instance-Weighting
Cross-lingual text classification (CLTC) is a challenging task made even harder still due to the lack of labeled data in low-resource languages. In this paper, we propose zero-shot instance-weighting, a general model-agn…
Classificationtext-classificationText ClassificationZero-Shot LearningTF-CADE: Foreground-Concentrated Text-Video Alignment for Zero-Shot Temporal Action Detection
Zero-Shot Temporal Action Detection (ZSTAD) aims to lo- calize and recognize action instances from unseen action categories in untrimmed videos. Although existing meth- ods have shown effectiveness by advancing architect…
Action DetectionVideo AlignmentSelf-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning
There is a rising interest in further exploring the zero-shot learning potential of large pre-trained language models (PLMs). A new paradigm called data-generation-based zero-shot learning has achieved impressive success…
text-classificationText Classificationzero-shot-classificationZero-Shot Learning