Over-Parameterization and Generalization in Audio Classification
Convolutional Neural Networks (CNNs) have been dominating classification tasks in various domains, such as machine vision, machine listening, and natural language processing. In machine listening, while generally exhibiting very good generalization capabilities, CNNs are sensitive to the specific audio recording device used, which has been recognized as a substantial problem in the acoustic scene classification (DCASE) community. In this study, we investigate the relationship between over-parameterization of acoustic scene classification models, and their resulting generalization abilities. Specifically, we test scaling CNNs in width and depth, under different conditions. Our results indicate that increasing width improves generalization to unseen devices, even without an increase in the number of parameters.
Code (0)
등록된 구현이 없습니다.
Tasks
Acoustic Scene ClassificationAudio ClassificationClassificationScene ClassificationSimilar Papers 제목 키워드 기반
ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks
We introduce an approach to training a given compact network. To this end, we leverage over-parameterization, which typically improves both neural network optimization and generalization. Specifically, we propose to expa…
General Classificationimage-classificationImage ClassificationKnowledge Distillation+4Analysis of Catastrophic Forgetting for Random Orthogonal Transformation Tasks in the Overparameterized Regime
Overparameterization is known to permit strong generalization performance in neural networks. In this work, we provide an initial theoretical analysis of its effect on catastrophic forgetting in a continual learning setu…
Continual Learningimage-classificationImage ClassificationPermuted-MNISTThe Price of Robustness: Stable Classifiers Need Overparameterization
The relationship between overparameterization, stability, and generalization remains incompletely understood in the setting of discontinuous classifiers. We address this gap by establishing a generalization bound for fin…
End-to-End Audio Strikes Back: Boosting Augmentations Towards An Efficient Audio Classification Network
While efficient architectures and a plethora of augmentations for end-to-end image classification tasks have been suggested and heavily investigated, state-of-the-art techniques for audio classifications still rely on nu…
Audio ClassificationClassificationEnvironmental Sound Classificationimage-classification+3Over-parameterization Improves Generalization in the XOR Detection Problem
Empirical evidence suggests that neural networks with ReLU activations generalize better with over-parameterization. However, there is currently no theoretical analysis that explains this observation. In this work, we st…