Compression of Acoustic Event Detection Models With Quantized Distillation

2019-07-01 · Bowen Shi, Ming Sun, Chieh-Chi Kao, Viktor Rozgic, Spyros Matsoukas, Chao Wang

Acoustic Event Detection (AED), aiming at detecting categories of events based on audio signals, has found application in many intelligent systems. Recently deep neural network significantly advances this field and reduces detection errors to a large scale. However how to efficiently execute deep models in AED has received much less attention. Meanwhile state-of-the-art AED models are based on large deep models, which are computational demanding and challenging to deploy on devices with constrained computational resources. In this paper, we present a simple yet effective compression approach which jointly leverages knowledge distillation and quantization to compress larger network (teacher model) into compact network (student model). Experimental results show proposed technique not only lowers error rate of original compact network by 15% through distillation but also further reduces its model size to a large extent (2% of teacher, 12% of full-precision student) through quantization.

📄 PDF Abstract BibTeX arXiv:1907.00873

Code (0)

등록된 구현이 없습니다.

Tasks

Event DetectionKnowledge Distillation Quantization

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Joint framework with deep feature distillation and adaptive focal loss for weakly supervised audio tagging and acoustic event detection

2021-03-23 · Yunhao Liang, Yanhua Long, Yijie Li, Jiaen Liang 외

A good joint training framework is very helpful to improve the performances of weakly supervised audio tagging (AT) and acoustic event detection (AED) simultaneously. In this study, we propose three methods to improve th…

Audio Tagging Event Detection

Semi-supervised Acoustic Event Detection based on tri-training

2019-04-29 · Bowen Shi, Ming Sun, Chieh-Chi Kao, Viktor Rozgic 외

This paper presents our work of training acoustic event detection (AED) models using unlabeled dataset. Recent acoustic event detectors are based on large-scale neural networks, which are typically trained with huge amou…

Event Detection Knowledge Distillation

Compression of Acoustic Event Detection Models with Low-rank Matrix Factorization and Quantization Training

2019-05-02 · NIPS Workshop CDNNRIA 2018 · Bowen Shi, Ming Sun, Chieh-Chi Kao, Viktor Rozgic 외

In this paper, we present a compression approach based on the combination of low-rank matrix factorization and quantization training, to reduce complexity for neural network based acoustic event detection (AED) models. O…

Event Detection Quantization

Model compression via distillation and quantization

2018-02-15 · ICLR 2018 1 · Antonio Polino, Razvan Pascanu, Dan Alistarh

Deep neural networks (DNNs) continue to make significant advances, solving tasks from image classification to translation or reinforcement learning. One aspect of the field receiving considerable attention is efficiently…

image-classification model Model Compression Quantization+1

Cross-task pre-training for on-device acoustic scene classification

2019-10-22 · Ruixiong Zhang, Wei Zou, Xiangang Li

Acoustic scene classification (ASC) and acoustic event detection (AED) are different but related tasks. Acoustic events can provide useful information for recognizing acoustic scenes. However, most of the datasets are pr…

Acoustic Scene Classification Classification Event Detection General Classification+1