Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions

2021-05-28 · Shanshan Wang, Toni Heittola, Annamaria Mesaros, Tuomas Virtanen

This paper presents the details of the Audio-Visual Scene Classification task in the DCASE 2021 Challenge (Task 1 Subtask B). The task is concerned with classification using audio and video modalities, using a dataset of synchronized recordings. This task has attracted 43 submissions from 13 different teams around the world. Among all submissions, more than half of the submitted systems have better performance than the baseline. The common techniques among the top systems are the usage of large pretrained models such as ResNet or EfficientNet which are trained for the task-specific problem. Fine-tuning, transfer learning, and data augmentation techniques are also employed to boost the performance. More importantly, multi-modal methods using both audio and video are employed by all the top 5 teams. The best system among all achieved a logloss of 0.195 and accuracy of 93.8%, compared to the baseline system with logloss of 0.662 and accuracy of 77.1%.

📄 PDF Abstract BibTeX arXiv:2105.13675

Code (0)

등록된 구현이 없습니다.

Tasks

Classification Data Augmentation Scene Classification Transfer Learning

Methods 이 논문이 사용한 방법론

Pointwise Convolution Pointwise Convolution is a type of convolution that uses a 1x1 kernel: a kernel that iterates through every single point. This…

Depthwise Convolution Depthwise Convolution is a type of convolution where we apply a single convolutional filter for each input channel. In the regular 2D…

Residual Connection 설명 없음

Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Kaiming Initialization 설명 없음

Sigmoid Activation 설명 없음

Depthwise Separable Convolution While standard convolution performs the channelwise and spatial-wise computation in one step, Depthwise Separable Convolution …

Residual Block Residual Blocks are skip-connection blocks that learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. They were introduced…

Stereo Sound Event Localization and Detection with Onscreen/offscreen Classification

2025-07-16 · Kazuki Shimada, Archontis Politis, Iran R. Roman, Parthasaarathy Sudarsanam 외 arxiv

This paper presents the objective, dataset, baseline, and metrics of Task 3 of the DCASE2025 Challenge on sound event localization and detection (SELD). In previous editions, the challenge used four-channel audio formats…

Sound Event Localization and Detection

A study on joint modeling and data augmentation of multi-modalities for audio-visual scene classification

2022-03-07 · Qing Wang, Jun Du, Siyuan Zheng, Yunqing Li 외

In this paper, we propose two techniques, namely joint modeling and data augmentation, to improve system performances for audio-visual scene classification (AVSC). We employ pre-trained networks trained only on image dat…

Data Augmentation Scene Classification

Sound Event Detection in Synthetic Audio: Analysis of the DCASE 2016 Task Results

2017-11-15 · Grégoire Lafay, Emmanouil Benetos, Mathieu Lagrange

As part of the 2016 public evaluation challenge on Detection and Classification of Acoustic Scenes and Events (DCASE 2016), the second task focused on evaluating sound event detection systems using synthetic mixtures of …

Event Detection General Classification Sound Event Detection

Exploring Audio-Visual Information Fusion for Sound Event Localization and Detection In Low-Resource Realistic Scenarios

2024-06-21 · Ya Jiang, Qing Wang, Jun Du, Maocheng Hu 외

This study presents an audio-visual information fusion approach to sound event localization and detection (SELD) in low-resource scenarios. We aim at utilizing audio and video modality information through cross-modal lea…

Data Augmentation Sound Event Localization and Detection

On The Effect Of Coding Artifacts On Acoustic Scene Classification

2021-12-09 · Nagashree K. S. Rao, Nils Peters

Previous DCASE challenges contributed to an increase in the performance of acoustic scene classification systems. State-of-the-art classifiers demand significant processing capabilities and memory which is challenging fo…

Acoustic Scene Classification Audio Compression Classification Scene Classification