Sentiment-Aware Automatic Speech Recognition pre-training for enhanced Speech Emotion Recognition
We propose a novel multi-task pre-training method for Speech Emotion Recognition (SER). We pre-train SER model simultaneously on Automatic Speech Recognition (ASR) and sentiment classification tasks to make the acoustic ASR model more ``emotion aware''. We generate targets for the sentiment classification using text-to-sentiment model trained on publicly available data. Finally, we fine-tune the acoustic ASR on emotion annotated speech data. We evaluated the proposed approach on the MSP-Podcast dataset, where we achieved the best reported concordance correlation coefficient (CCC) of 0.41 for valence prediction.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)ClassificationEmotion RecognitionSentiment AnalysisSentiment ClassificationSpeech Emotion Recognitionspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Incorporating End-to-End Speech Recognition Models for Sentiment Analysis
Previous work on emotion recognition demonstrated a synergistic effect of combining several modalities such as auditory, visual, and transcribed text to estimate the affective state of a speaker. Among these, the linguis…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Emotion RecognitionSentiment Analysis+3Leveraging Pre-trained Language Model for Speech Sentiment Analysis
In this paper, we explore the use of pre-trained language models to learn sentiment information of written texts for speech sentiment analysis. First, we investigate how useful a pre-trained language model would be in a …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language ModelingLanguage Modelling+4Generative Context-aware Fine-tuning of Self-supervised Speech Models
When performing tasks like automatic speech recognition or spoken language understanding for a given utterance, access to preceding text or audio provides contextual information can improve performance. Considering the r…
Automatic Speech Recognitionnamed-entity-recognitionNamed Entity RecognitionSentence+4Speech Emotion Recognition via Entropy-Aware Score Selection
In this paper, we propose a multimodal framework for speech emotion recognition that leverages entropy-aware score selection to combine speech and textual predictions. The proposed method integrates a primary pipeline th…
Speech Emotion RecognitionSentiment AnalysisSentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors
Multimodal sentiment analysis has attracted increasing attention and lots of models have been proposed. However, the performance of the state-of-the-art models decreases sharply when they are deployed in the real world. …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Multimodal Sentiment AnalysisPosition+4