Generalized Zero-Shot Learning using Multimodal Variational Auto-Encoder with Semantic Concepts
With the ever-increasing amount of data, the central challenge in multimodal learning involves limitations of labelled samples. For the task of classification, techniques such as meta-learning, zero-shot learning, and few-shot learning showcase the ability to learn information about novel classes based on prior knowledge. Recent techniques try to learn a cross-modal mapping between the semantic space and the image space. However, they tend to ignore the local and global semantic knowledge. To overcome this problem, we propose a Multimodal Variational Auto-Encoder (M-VAE) which can learn the shared latent space of image features and the semantic space. In our approach we concatenate multimodal data to a single embedding before passing it to the VAE for learning the latent space. We propose the use of a multi-modal loss during the reconstruction of the feature embedding through the decoder. Our approach is capable to correlating modalities and exploit the local and global semantic knowledge for novel sample predictions. Our experimental results using a MLP classifier on four benchmark datasets show that our proposed model outperforms the current state-of-the-art approaches for generalized zero-shot learning.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderFew-Shot LearningGeneralized Zero-Shot LearningMeta-LearningZero-Shot LearningSimilar Papers 제목 키워드 기반
Cross-Linked Variational Autoencoders for Generalized Zero-Shot Learning
Most approaches in generalized zero-shot learning rely on cross-modal mapping between an image feature space and a class embedding space or on generating artificial image features. However, learning a shared cross-modal …
Few-Shot LearningGeneralized Zero-Shot LearningZero-Shot LearningGeneralized Zero- and Few-Shot Learning via Aligned Variational Autoencoders
Many approaches in generalized zero-shot learning rely on cross-modal mapping between the image feature space and the class embedding space. As labeled images are expensive, one direction is to augment the dataset by gen…
Few-Shot LearningGeneralized Few-Shot LearningGeneralized Zero-Shot LearningZero-Shot Learning+1Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders
Many approaches in generalized zero-shot learning rely on cross-modal mapping between the image feature space and the class embedding space. As labeled images are expensive, one direction is to augment the dataset by gen…
Few-Shot LearningGeneralized Few-Shot LearningGeneralized Zero-Shot LearningGeneralized Zero-Shot Learning - Unseen+2Multimodal Generalized Zero Shot Learning for Gleason Grading using Self-Supervised Learning
Gleason grading from histopathology images is essential for accurate prostate cancer (PCa) diagnosis. Since such images are obtained after invasive tissue resection quick diagnosis is challenging under the existing parad…
Generalized Zero-Shot LearningSelf-Supervised LearningZero-Shot LearningGeneralized Zero-Shot Domain Adaptation via Coupled Conditional Variational Autoencoders
Domain adaptation approaches aim to exploit useful information from the source domain where supervised learning examples are easier to obtain to address a learning problem in the target domain where there is no or limite…
Domain AdaptationGeneralized Zero-Shot LearningZero-Shot Learning