Speech Emotion Recognition with Global-Aware Fusion on Multi-scale Feature Representation
Speech Emotion Recognition (SER) is a fundamental task to predict the emotion label from speech data. Recent works mostly focus on using convolutional neural networks~(CNNs) to learn local attention map on fixed-scale feature representation by viewing time-varied spectral features as images. However, rich emotional feature at different scales and important global information are not able to be well captured due to the limits of existing CNNs for SER. In this paper, we propose a novel GLobal-Aware Multi-scale (GLAM) neural network (The code is available at https://github.com/lixiangucas01/GLAM) to learn multi-scale feature representation with global-aware fusion module to attend emotional information. Specifically, GLAM iteratively utilizes multiple convolutional kernels with different scales to learn multiple feature representation. Then, instead of using attention-based methods, a simple but effective global-aware fusion module is applied to grab most important emotional information globally. Experiments on the benchmark corpus IEMOCAP over four emotions demonstrates the superiority of our proposed model with 2.5% to 4.5% improvements on four common metrics compared to previous state-of-the-art approaches.
Code (1)
Tasks
Emotion RecognitionSpeech Emotion RecognitionSimilar Papers 제목 키워드 기반
Seeing is Believing: Emotion-Aware Audio-Visual Language Modeling for Expressive Speech Generation
We present an Audio-Visual Language Model (AVLM) for expressive speech generation by integrating full-face visual cues into a pre-trained expressive speech model. We explore multiple visual encoders and multimodal fusion…
Emotion RecognitionSpeech Emotion Recognition Considering Local Dynamic Features
Recently, increasing attention has been directed to the study of the speech emotion recognition, in which global acoustic features of an utterance are mostly used to eliminate the content differences. However, the expres…
Emotion RecognitionSpeech Emotion RecognitionSpeech Emotion Recognition Based on Multi-feature and Multi-lingual Fusion
A speech emotion recognition algorithm based on multi-feature and Multi-lingual fusion is proposed in order to resolve low recognition accuracy caused by lack of large speech dataset and low robustness of acoustic featur…
Emotion RecognitionSpeech Emotion RecognitionSpeech Emotion Recognition via Entropy-Aware Score Selection
In this paper, we propose a multimodal framework for speech emotion recognition that leverages entropy-aware score selection to combine speech and textual predictions. The proposed method integrates a primary pipeline th…
Speech Emotion RecognitionSentiment AnalysisSentiment-Aware Automatic Speech Recognition pre-training for enhanced Speech Emotion Recognition
We propose a novel multi-task pre-training method for Speech Emotion Recognition (SER). We pre-train SER model simultaneously on Automatic Speech Recognition (ASR) and sentiment classification tasks to make the acoustic …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)ClassificationEmotion Recognition+5