Multi-label Zero-shot Classification by Learning to Transfer from External Knowledge
Multi-label zero-shot classification aims to predict multiple unseen class labels for an input image. It is more challenging than its single-label counterpart. On one hand, the unconstrained number of labels assigned to each image makes the model more easily overfit to those seen classes. On the other hand, there is a large semantic gap between seen and unseen classes in the existing multi-label classification datasets. To address these difficult issues, this paper introduces a novel multi-label zero-shot classification framework by learning to transfer from external knowledge. We observe that ImageNet is commonly used to pretrain the feature extractor and has a large and fine-grained label space. This motivates us to exploit it as external knowledge to bridge the seen and unseen classes and promote generalization. Specifically, we construct a knowledge graph including not only classes from the target dataset but also those from ImageNet. Since ImageNet labels are not available in the target dataset, we propose a novel PosVAE module to infer their initial states in the extended knowledge graph. Then we design a relational graph convolutional network (RGCN) to propagate information among classes and achieve knowledge transfer. Experimental results on two benchmark datasets demonstrate the effectiveness of the proposed approach.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral ClassificationMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONTransfer Learningzero-shot-classificationZero-Shot LearningSimilar Papers 제목 키워드 기반
Multi-Label Zero-Shot Learning with Transfer-Aware Label Embedding Projection
Zero-shot learning transfers knowledge from seen classes to novel unseen classes to reduce human labor of labelling data for building new classifiers. Much effort on zero-shot learning however has focused on the standard…
image-classificationImage ClassificationMulti-Label Image ClassificationMulti-label zero-shot learning+2Realistic Zero-Shot Cross-Lingual Transfer in Legal Topic Classification
We consider zero-shot cross-lingual transfer in legal topic classification using the recent MultiEURLEX dataset. Since the original dataset contains parallel documents, which is unrealistic for zero-shot cross-lingual tr…
Cross-Lingual TransferTopic ClassificationTranslationZero-Shot Cross-Lingual TransferRealistic Zero-Shot Cross-Lingual Transfer in Legal Topic Classification
We consider zero-shot cross-lingual transfer in legal topic classification using the recent Multi-EURLEX dataset. Since the original dataset contains parallel documents, which is unrealistic for zero-shot cross-lingual t…
ClassificationCross-Lingual TransferTopic ClassificationTranslation+1Zero-shot Learning and Knowledge Transfer in Music Classification and Tagging
Music classification and tagging is conducted through categorical supervised learning with a fixed set of labels. In principle, this cannot make predictions on unseen labels. Zero-shot learning is an approach to solve th…
ClassificationGeneral ClassificationMusic ClassificationTransfer Learning+1Multi-label Zero-Shot Audio Classification with Temporal Attention
Zero-shot learning models are capable of classifying new classes by transferring knowledge from the seen classes using auxiliary information. While most of the existing zero-shot learning methods focused on single-label …
Audio ClassificationClassificationZero-shot Audio Classificationzero-shot-classification+1