Exploring Self-Supervised Contrastive Learning of Spatial Sound Event Representation
In this study, we present a simple multi-channel framework for contrastive learning (MC-SimCLR) to encode 'what' and 'where' of spatial audios. MC-SimCLR learns joint spectral and spatial representations from unlabeled spatial audios, thereby enhancing both event classification and sound localization in downstream tasks. At its core, we propose a multi-level data augmentation pipeline that augments different levels of audio features, including waveforms, Mel spectrograms, and generalized cross-correlation (GCC) features. In addition, we introduce simple yet effective channel-wise augmentation methods to randomly swap the order of the microphones and mask Mel and GCC channels. By using these augmentations, we find that linear layers on top of the learned representation significantly outperform supervised models in terms of both event classification accuracy and localization error. We also perform a comprehensive analysis of the effect of each augmentation method and a comparison of the fine-tuning performance using different amounts of labeled data.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningData AugmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multi-organ Self-supervised Contrastive Learning for Breast Lesion Segmentation
Self-supervised learning has proven to be an effective way to learn representations in domains where annotated labels are scarce, such as medical imaging. A widely adopted framework for this purpose is contrastive learni…
Contrastive LearningLesion SegmentationSelf-Supervised LearningTowards Objective Obstetric Ultrasound Assessment: Contrastive Representation Learning for Fetal Movement Detection
Accurate fetal movement (FM) detection is essential for assessing prenatal health, as abnormal movement patterns can indicate underlying complications such as placental dysfunction or fetal distress. Traditional methods,…
Self-Supervised LearningRepresentation LearningContrastive LearningExploring Self-Supervised Representation Ensembles for COVID-19 Cough Classification
The usage of smartphone-collected respiratory sound, trained with deep learning models, for detecting and classifying COVID-19 becomes popular recently. It removes the need for in-person testing procedures especially for…
ClassificationCough ClassificationDiagnosticSelf-Supervised LearningSelf-supervised Contrastive Learning for Audio-Visual Action Recognition
The underlying correlation between audio and visual modalities can be utilized to learn supervised information for unlabeled videos. In this paper, we propose an end-to-end self-supervised framework named Audio-Visual Co…
Action RecognitionContrastive LearningSelf-Supervised Action RecognitionUnsupervised Contrastive Learning of Sound Event Representations
Self-supervised representation learning can mitigate the limitations in recognition tasks with few manually labeled data but abundant unlabeled data---a common scenario in sound event research. In this work, we explore u…
Contrastive LearningLinear evaluationRepresentation Learning