Data-Free Generalized Zero-Shot Learning
Deep learning models have the ability to extract rich knowledge from large-scale datasets. However, the sharing of data has become increasingly challenging due to concerns regarding data copyright and privacy. Consequently, this hampers the effective transfer of knowledge from existing data to novel downstream tasks and concepts. Zero-shot learning (ZSL) approaches aim to recognize new classes by transferring semantic knowledge learned from base classes. However, traditional generative ZSL methods often require access to real images from base classes and rely on manually annotated attributes, which presents challenges in terms of data restrictions and model scalability. To this end, this paper tackles a challenging and practical problem dubbed as data-free zero-shot learning (DFZSL), where only the CLIP-based base classes data pre-trained classifier is available for zero-shot classification. Specifically, we propose a generic framework for DFZSL, which consists of three main components. Firstly, to recover the virtual features of the base data, we model the CLIP features of base class images as samples from a von Mises-Fisher (vMF) distribution based on the pre-trained classifier. Secondly, we leverage the text features of CLIP as low-cost semantic information and propose a feature-language prompt tuning (FLPT) method to further align the virtual image features and textual features. Thirdly, we train a conditional generative model using the well-aligned virtual image features and corresponding semantic text features, enabling the generation of new classes features and achieve better zero-shot generalization. Our framework has been evaluated on five commonly used benchmarks for generalized ZSL, as well as 11 benchmarks for the base-to-new ZSL. The results demonstrate the superiority and effectiveness of our approach. Our code is available in https://github.com/ylong4/DFZSL
Code (1)
Tasks
Generalized Zero-Shot Learningzero-shot-classificationZero-shot GeneralizationZero-Shot LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning the Redundancy-free Features for Generalized Zero-Shot Object Recognition
Zero-shot object recognition or zero-shot learning aims to transfer the object recognition ability among the semantically related categories, such as fine-grained animal or bird species. However, the images of different …
Generalized Zero-Shot LearningObjectObject RecognitionZero-Shot LearningFREE: Feature Refinement for Generalized Zero-Shot Learning
Generalized zero-shot learning (GZSL) has achieved significant progress, with many efforts dedicated to overcoming the problems of visual-semantic domain gap and seen-unseen bias. However, most existing methods directly …
Generalized Zero-Shot LearningZero-Shot LearningGeneralized Zero-Shot Learning for Action Recognition with Web-Scale Video Data
Action recognition in surveillance video makes our life safer by detecting the criminal events or predicting violent emergencies. However, efficient action recognition is not free of difficulty. First, there are so many …
Action RecognitionGeneralized Zero-Shot LearningTemporal Action LocalizationZero-Shot LearningModel Selection for Generalized Zero-shot Learning
In the problem of generalized zero-shot learning, the datapoints from unknown classes are not available during training. The main challenge for generalized zero-shot learning is the unbalanced data distribution which mak…
Generalized Zero-Shot LearningGenerative Adversarial NetworkmodelModel Selection+1A Unified approach for Conventional Zero-shot, Generalized Zero-shot and Few-shot Learning
Prevalent techniques in zero-shot learning do not generalize well to other related problem scenarios. Here, we present a unified approach for conventional zero-shot, generalized zero-shot and few-shot learning problems. …
Few-Shot LearningGeneralized Zero-Shot LearningOne-Shot LearningZero-Shot Learning