Zero-Shot Audio Classification using Image Embeddings
Supervised learning methods can solve the given problem in the presence of a large set of labeled data. However, the acquisition of a dataset covering all the target classes typically requires manual labeling which is expensive and time-consuming. Zero-shot learning models are capable of classifying the unseen concepts by utilizing their semantic information. The present study introduces image embeddings as side information on zero-shot audio classification by using a nonlinear acoustic-semantic projection. We extract the semantic image representations from the Open Images dataset and evaluate the performance of the models on an audio subset of AudioSet using semantic information in different domains; image, audio, and textual. We demonstrate that the image embeddings can be used as semantic information to perform zero-shot audio classification. The experimental results show that the image and textual embeddings display similar performance both individually and together. We additionally calculate the semantic acoustic embeddings from the test samples to provide an upper limit to the performance. The results show that the classification performance is highly sensitive to the semantic relation between test and training classes and textual and image embeddings can reach up to the semantic acoustic embeddings when the seen and unseen classes are semantically similar.
Code (0)
등록된 구현이 없습니다.
Tasks
Audio ClassificationClassificationZero-shot Audio ClassificationZero-Shot LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Audio Classification by Transitioning from Zero- to Few-Shot
State-of-the-art audio classification often employs a zero-shot approach, which involves comparing audio embeddings with embeddings from text describing the respective audio class. These embeddings are usually generated …
Contrastive LearningAudio ClassificationZero-Shot Audio Classification Based on Class Label Embeddings
This paper proposes a zero-shot learning approach for audio classification based on the textual information about class labels without any audio samples from target classes. We propose an audio classification system buil…
Audio ClassificationClassificationGeneral ClassificationZero-shot Audio Classification+1Zero-Shot Audio Classification with Factored Linear and Nonlinear Acoustic-Semantic Projections
In this paper, we study zero-shot learning in audio classification through factored linear and nonlinear acoustic-semantic projections between audio instances and sound classes. Zero-shot learning in audio classification…
Audio ClassificationClassificationGeneral ClassificationZero-shot Audio Classification+1Zero-Shot Audio Classification via Semantic Embeddings
In this paper, we study zero-shot learning in audio classification via semantic embeddings extracted from textual labels and sentence descriptions of sound classes. Our goal is to obtain a classifier that is capable of r…
Audio ClassificationClassificationGeneral ClassificationSentence+3AVGZSLNet: Audio-Visual Generalized Zero-Shot Learning by Reconstructing Label Features from Multi-Modal Embeddings
In this paper, we propose a novel approach for generalized zero-shot learning in a multi-modal setting, where we have novel classes of audio/video during testing that are not seen during training. We use the semantic rel…
DecoderGeneralized Zero-Shot LearningGZSL Video ClassificationRetrieval+3