Zero-Shot Audio Classification Based on Class Label Embeddings

2019-05-06 · Huang Xie, Tuomas Virtanen

This paper proposes a zero-shot learning approach for audio classification based on the textual information about class labels without any audio samples from target classes. We propose an audio classification system built on the bilinear model, which takes audio feature embeddings and semantic class label embeddings as input, and measures the compatibility between an audio feature embedding and a class label embedding. We use VGGish to extract audio feature embeddings from audio recordings. We treat textual labels as semantic side information of audio classes, and use Word2Vec to generate class label embeddings. Results on the ESC-50 dataset show that the proposed system can perform zero-shot audio classification with small training dataset. It can achieve accuracy (26 % on average) better than random guess (10 %) on each audio category. Particularly, it reaches up to 39.7 % for the category of natural audio classes.

📄 PDF Abstract BibTeX arXiv:1905.01926

Code (0)

등록된 구현이 없습니다.

Tasks

Audio Classification Classification General ClassificationZero-shot Audio ClassificationZero-Shot Learning

Multi-label Zero-Shot Audio Classification with Temporal Attention

2024-08-31 · Duygu Dogan, Huang Xie, Toni Heittola, Tuomas Virtanen

Zero-shot learning models are capable of classifying new classes by transferring knowledge from the seen classes using auxiliary information. While most of the existing zero-shot learning methods focused on single-label …

Audio Classification Classification Zero-shot Audio Classification zero-shot-classification+1

A sound description: Exploring prompt templates and class descriptions to enhance zero-shot audio classification

2024-09-19 · Michel Olvera, Paraskevas Stamatiadis, Slim Essid

Audio-text models trained via contrastive learning offer a practical approach to perform audio classification through natural language prompts, such as "this is a sound of" followed by category names. In this work, we ex…

Audio Classification Classification Contrastive Learning Prompt Engineering+1

TSPE: Task-Specific Prompt Ensemble for Improved Zero-Shot Audio Classification

2024-12-31 · Nishit Anand, Ashish Seth, Ramani Duraiswami, Dinesh Manocha

Audio-language models (ALMs) excel in zero-shot audio classification, a task where models classify previously unseen audio clips at test time by leveraging descriptive natural language prompts. We introduce TSPE (Task-Sp…

Audio Classification Classification Descriptive Zero-shot Audio Classification

Adapting Language-Audio Models as Few-Shot Audio Learners

2023-05-28 · Jinhua Liang, Xubo Liu, Haohe Liu, Huy Phan 외

We presented the Treff adapter, a training-efficient adapter for CLAP, to boost zero-shot classification performance by making use of a small set of labelled data. Specifically, we designed CALM to retrieve the probabili…

Audio Classification Classification Few-Shot Learning zero-shot-classification+1

Zero-Shot Audio Classification via Semantic Embeddings

2020-11-24 · Huang Xie, Tuomas Virtanen

In this paper, we study zero-shot learning in audio classification via semantic embeddings extracted from textual labels and sentence descriptions of sound classes. Our goal is to obtain a classifier that is capable of r…

Audio Classification Classification General Classification Sentence+3