Multilingual Speech Emotion Recognition With Multi-Gating Mechanism and Neural Architecture Search
Speech emotion recognition (SER) classifies audio into emotion categories such as Happy, Angry, Fear, Disgust and Neutral. While Speech Emotion Recognition (SER) is a common application for popular languages, it continues to be a problem for low-resourced languages, i.e., languages with no pretrained speech-to-text recognition models. This paper firstly proposes a language-specific model that extract emotional information from multiple pre-trained speech models, and then designs a multi-domain model that simultaneously performs SER for various languages. Our multidomain model employs a multi-gating mechanism to generate unique weighted feature combination for each language, and also searches for specific neural network structure for each language through a neural architecture search module. In addition, we introduce a contrastive auxiliary loss to build more separable representations for audio data. Our experiments show that our model raises the state-of-the-art accuracy by 3% for German and 14.3% for French.
Code (0)
등록된 구현이 없습니다.
Tasks
Emotion RecognitionNeural Architecture SearchSpeech Emotion RecognitionSpeech-to-TextSimilar Papers 제목 키워드 기반
Large Language Models Meet Contrastive Learning: Zero-Shot Emotion Recognition Across Languages
Multilingual speech emotion recognition aims to estimate a speaker's emotional state using a contactless method across different languages. However, variability in voice characteristics and linguistic diversity poses sig…
Contrastive LearningDiversityEmotion RecognitionSpeech Emotion RecognitionSeQuiFi: Mitigating Catastrophic Forgetting in Speech Emotion Recognition with Sequential Class-Finetuning
In this work, we introduce SeQuiFi, a novel approach for mitigating catastrophic forgetting (CF) in speech emotion recognition (SER). SeQuiFi adopts a sequential class-finetuning strategy, where the model is fine-tuned i…
Continual LearningEmotion RecognitionSpeech Emotion RecognitionCross-lingual and Multilingual Speech Emotion Recognition on English and French
Research on multilingual speech emotion recognition faces the problem that most available speech corpora differ from each other in important ways, such as annotation methods or interaction scenarios. These inconsistencie…
Emotion RecognitionSpeech Emotion RecognitionCAMEO: Collection of Multilingual Emotional Speech Corpora
This paper presents CAMEO -- a curated collection of multilingual emotional speech datasets designed to facilitate research in emotion recognition and other speech-related tasks. The main objectives were to ensure easy a…
Emotion RecognitionSpeech Emotion RecognitionEnsembling Multilingual Pre-Trained Models for Predicting Multi-Label Regression Emotion Share from Speech
Speech emotion recognition has evolved from research to practical applications. Previous studies of emotion recognition from speech have focused on developing models on certain datasets like IEMOCAP. The lack of data in …
Emotion RecognitionEnsemble LearningSpeech Emotion Recognition