Developing a High-performance Framework for Speech Emotion Recognition in Naturalistic Conditions Challenge for Emotional Attribute Prediction
Speech emotion recognition (SER) in naturalistic conditions presents a significant challenge for the speech processing community. Challenges include disagreement in labeling among annotators and imbalanced data distributions. This paper presents a reproducible framework that achieves superior (top 1) performance in the Emotion Recognition in Naturalistic Conditions Challenge (IS25-SER Challenge) - Task 2, evaluated on the MSP-Podcast dataset. Our system is designed to tackle the aforementioned challenges through multimodal learning, multi-task learning, and imbalanced data handling. Specifically, our best system is trained by adding text embeddings, predicting gender, and including `Other'' (O) and `No Agreement'' (X) samples in the training set. Our system's results secured both first and second places in the IS25-SER Challenge, and the top performance was achieved by a simple two-system ensemble.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeEmotion RecognitionMulti-Task LearningSpeech Emotion RecognitionTask 2Similar Papers 제목 키워드 기반
EmoSpeech: A Corpus of Emotionally Rich and Contextually Detailed Speech Annotations
Advances in text-to-speech (TTS) technology have significantly improved the quality of generated speech, closely matching the timbre and intonation of the target speaker. However, due to the inherent complexity of human …
text-to-speechText to SpeechSpeech and Text-Based Emotion Recognizer
Affective computing is a field of study that focuses on developing systems and technologies that can understand, interpret, and respond to human emotions. Speech Emotion Recognition (SER), in particular, has got a lot of…
Data AugmentationEmotion RecognitionSpeech Emotion RecognitionEnsembling Multilingual Pre-Trained Models for Predicting Multi-Label Regression Emotion Share from Speech
Speech emotion recognition has evolved from research to practical applications. Previous studies of emotion recognition from speech have focused on developing models on certain datasets like IEMOCAP. The lack of data in …
Emotion RecognitionEnsemble LearningSpeech Emotion RecognitionEmoNews: A Spoken Dialogue System for Expressive News Conversations
We develop a task-oriented spoken dialogue system (SDS) that regulates emotional speech based on contextual cues to enable more empathetic news conversations. Despite advancements in emotional text-to-speech (TTS) techni…
Language ModelingLanguage ModellingLarge Language Modeltext-to-speech+1BLSP-Emo: Towards Empathetic Large Speech-Language Models
The recent release of GPT-4o showcased the potential of end-to-end multimodal models, not just in terms of low latency but also in their ability to understand and generate expressive speech with rich emotions. While the …
Emotion RecognitionInstruction FollowingLanguage ModelingLanguage Modelling+3