Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation
Foundation models have shown superior performance for speech emotion recognition (SER). However, given the limited data in emotion corpora, finetuning all parameters of large pre-trained models for SER can be both resource-intensive and susceptible to overfitting. This paper investigates parameter-efficient finetuning (PEFT) for SER. Various PEFT adaptors are systematically studied for both classification of discrete emotion categories and prediction of dimensional emotional attributes. The results demonstrate that the combination of PEFT methods surpasses full finetuning with a significant reduction in the number of trainable parameters. Furthermore, a two-stage adaptation strategy is proposed to adapt models trained on acted emotion data, which is more readily available, to make the model more adept at capturing natural emotional expressions. Both intra- and cross-corpus experiments validate the efficacy of the proposed approach in enhancing the performance on both the source and target domains.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-corpusDomain AdaptationEmotion RecognitionSpeech Emotion RecognitionSimilar Papers 제목 키워드 기반
Pre-Finetuning for Few-Shot Emotional Speech Recognition
Speech models have long been known to overfit individual speakers for many classification tasks. This leads to poor generalization in settings where the speakers are out-of-domain or out-of-distribution, as is common in …
Few-Shot Learningspeech-recognitionSpeech RecognitionTransfer LearningAn analysis of large speech models-based representations for speech emotion recognition
Large speech models-derived features have recently shown increased performance over signal-based features across multiple downstream tasks, even when the networks are not finetuned towards the target task. In this paper …
Emotion RecognitionSpeech Emotion RecognitionEmotion Recognition from Speech Using Wav2vec 2.0 Embeddings
Emotion recognition datasets are relatively small, making the use of the more sophisticated deep learning approaches challenging. In this work, we propose a transfer learning method for speech emotion recognition where f…
Emotion RecognitionSpeech Emotion Recognitionspeech-recognitionSpeech Recognition+1SeQuiFi: Mitigating Catastrophic Forgetting in Speech Emotion Recognition with Sequential Class-Finetuning
In this work, we introduce SeQuiFi, a novel approach for mitigating catastrophic forgetting (CF) in speech emotion recognition (SER). SeQuiFi adopts a sequential class-finetuning strategy, where the model is fine-tuned i…
Continual LearningEmotion RecognitionSpeech Emotion RecognitionExploring Acoustic Similarity in Emotional Speech and Music via Self-Supervised Representations
Emotion recognition from speech and music shares similarities due to their acoustic overlap, which has led to interest in transferring knowledge between these domains. However, the shared acoustic cues between speech and…
Domain AdaptationDomain GeneralizationEmotion RecognitionMusic Emotion Recognition+3