Bird Vocalization Embedding Extraction Using Self-Supervised Disentangled Representation Learning
This paper addresses the extraction of the bird vocalization embedding from the whole song level using disentangled representation learning (DRL). Bird vocalization embeddings are necessary for large-scale bioacoustic tasks, and self-supervised methods such as Variational Autoencoder (VAE) have shown their performance in extracting such low-dimensional embeddings from vocalization segments on the note or syllable level. To extend the processing level to the entire song instead of cutting into segments, this paper regards each vocalization as the generalized and discriminative part and uses two encoders to learn these two parts. The proposed method is evaluated on the Great Tits dataset according to the clustering performance, and the results outperform the compared pre-trained models and vanilla VAE. Finally, this paper analyzes the informative part of the embedding, further compresses its dimension, and explains the disentangled performance of bird vocalizations.
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningSimilar Papers 제목 키워드 기반
Directional Embedding Based Semi-supervised Framework For Bird Vocalization Segmentation
This paper proposes a data-efficient, semi-supervised, two-pass framework for segmenting bird vocalizations. The framework utilizes a binary classification model to categorize frames of an input audio recording into the …
Binary ClassificationGeneral ClassificationBalanced Deep CCA for Bird Vocalization Detection
Event detection improves when events are captured by two different modalities rather than just one. But to train detection systems on multiple modalities is challenging, in particular when there is abundance of unlabelle…
Event DetectionSelf-Supervised LearningEstimating the Repertoire Size in Birds using Unsupervised Clustering techniques
Birds produce multiple types of vocalizations that, together, constitute a vocal repertoire. For some species, the repertoire size is of importance because it informs us about their brain capacity, territory size or soci…
ClusteringData-Efficient Self-Supervised Algorithms for Fine-Grained Birdsong Analysis
Research in bioacoustics, neuroscience, and linguistics often uses birdsong as a proxy to acquire knowledge across diverse areas. This requires audio models to annotate and parse the birdsong. Developing such models requ…
Self-Supervised LearningOnline ClusteringData AugmentationAutomatic recognition of element classes and boundaries in the birdsong with variable sequences
Researches on sequential vocalization often require analysis of vocalizations in long continuous sounds. In such studies as developmental ones or studies across generations in which days or months of vocalizations must b…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Boundary DetectionGeneral Classification+2