Explaining Spectrograms in Machine Learning: A Study on Neural Networks for Speech Classification
This study investigates discriminative patterns learned by neural networks for accurate speech classification, with a specific focus on vowel classification tasks. By examining the activations and features of neural networks for vowel classification, we gain insights into what the networks "see" in spectrograms. Through the use of class activation mapping, we identify the frequencies that contribute to vowel classification and compare these findings with linguistic knowledge. Experiments on a American English dataset of vowels showcases the explainability of neural networks and provides valuable insights into the causes of misclassifications and their characteristics when differentiating them from unvoiced speech. This study not only enhances our understanding of the underlying acoustic cues in vowel classification but also offers opportunities for improving speech recognition by bridging the gap between abstract representations in neural networks and established linguistic knowledge
Code (1)
Tasks
Classificationspeech-recognitionSpeech RecognitionVowel ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Transformer-based Sequence Labeling for Audio Classification based on MFCCs
Audio classification is vital in areas such as speech and music recognition. Feature extraction from the audio signal, such as Mel-Spectrograms and MFCCs, is a critical step in audio classification. These features are tr…
Audio ClassificationClassificationLeveraging AM and FM Rhythm Spectrograms for Dementia Classification and Assessment
This study explores the potential of Rhythm Formant Analysis (RFA) to capture long-term temporal modulations in dementia speech. Specifically, we introduce RFA-derived rhythm spectrograms as novel features for dementia c…
ClassificationregressionRhythmSpeech Emotion Recognition with Dual-Sequence LSTM Architecture
Speech Emotion Recognition (SER) has emerged as a critical component of the next generation human-machine interfacing technologies. In this work, we propose a new dual-level model that predicts emotions based on both MFC…
Emotion RecognitionSpeech Emotion RecognitionOn combining acoustic and modulation spectrograms in an attention LSTM-based system for speech intelligibility level classification
Speech intelligibility can be affected by multiple factors, such as noisy environments, channel distortions or physiological issues. In this work, we deal with the problem of automatic prediction of the speech intelligib…
Analysis of French Phonetic Idiosyncrasies for Accent Recognition
Speech recognition systems have made tremendous progress since the last few decades. They have developed significantly in identifying the speech of the speaker. However, there is a scope of improvement in speech recognit…
Multi-class Classificationspeech-recognitionSpeech Recognition