Exploring Meta Information for Audio-based Zero-shot Bird Classification
Advances in passive acoustic monitoring and machine learning have led to the procurement of vast datasets for computational bioacoustic research. Nevertheless, data scarcity is still an issue for rare and underrepresented species. This study investigates how meta-information can improve zero-shot audio classification, utilising bird species as an example case study due to the availability of rich and diverse meta-data. We investigate three different sources of metadata: textual bird sound descriptions encoded via (S)BERT, functional traits (AVONET), and bird life-history (BLH) characteristics. As audio features, we extract audio spectrogram transformer (AST) embeddings and project them to the dimension of the auxiliary information by adopting a single linear layer. Then, we employ the dot product as compatibility function and a standard zero-shot learning ranking hinge loss to determine the correct class. The best results are achieved by concatenating the AVONET and BLH features attaining a mean unweighted F1-score of .233 over five different test sets with 8 to 10 classes.
Code (1)
Tasks
Audio ClassificationZero-shot Audio ClassificationZero-Shot LearningSimilar Papers 제목 키워드 기반
A sound description: Exploring prompt templates and class descriptions to enhance zero-shot audio classification
Audio-text models trained via contrastive learning offer a practical approach to perform audio classification through natural language prompts, such as "this is a sound of" followed by category names. In this work, we ex…
Audio ClassificationClassificationContrastive LearningPrompt Engineering+1Hyperbolic Audio-visual Zero-shot Learning
Audio-visual zero-shot learning aims to classify samples consisting of a pair of corresponding audio and video sequences from classes that are not present during training. An analysis of the audio-visual data reveals a l…
GZSL Video ClassificationZero-Shot LearningExtremely Simple Out-of-distribution Detection for Audio-visual Generalized Zero-shot Learning
Zero-shot Learning(ZSL) attains knowledge transfer from seen classes to unseen classes by exploring auxiliary category information, which is a promising yet difficult research topic. In this field, Audio-Visual Generaliz…
Binary ClassificationGeneralized Zero-Shot LearningOut-of-Distribution DetectionOut of Distribution (OOD) Detection+2When Denoising Hinders: Revisiting Zero-Shot ASR with SAM-Audio and Whisper
Recent advances in automatic speech recognition (ASR) and speech enhancement have led to a widespread assumption that improving perceptual audio quality should directly benefit recognition accuracy. In this work, we rigo…
Speech RecognitionSpeech EnhancementGeneralized zero-shot audio-to-intent classification
Spoken language understanding systems using audio-only data are gaining popularity, yet their ability to handle unseen intents remains limited. In this study, we propose a generalized zero-shot audio-to-intent classifica…
ClassificationGoal-Oriented Dialogintent-classificationIntent Classification+4