Do Multimodal Emotion Recognition Models Tackle Ambiguity?
Most databases used for emotion recognition assign a single emotion to data samples. This does not match with the complex nature of emotions: we can feel a wide range of emotions throughout our lives with varying degrees of intensity. We may even experience multiple emotions at once. Furthermore, each person physically expresses emotions differently, which makes emotion recognition even more challenging: we call this emotional ambiguity. This paper investigates the problem as a review of ambiguity in multimodal emotion recognition models. To lay the groundwork, the main representations of emotions along with solutions for incorporating ambiguity are described, followed by a brief overview of ambiguity representation in multimodal databases. Thereafter, only models trained on a database that incorporates ambiguity have been studied in this paper. We conclude that although databases provide annotations with ambiguity, most of these models do not fully exploit them, showing that there is still room for improvement in multimodal emotion recognition systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Emotion RecognitionMultimodal Emotion RecognitionSimilar Papers 제목 키워드 기반
Multimodal Speech Emotion Recognition and Ambiguity Resolution
Identifying emotion from speech is a non-trivial task pertaining to the ambiguous definition of emotion itself. In this work, we adopt a feature-engineering based approach to tackle the task of speech emotion recognition…
BIG-bench Machine LearningEmotion RecognitionFeature EngineeringMulti-class Classification+2MEmoBERT: Pre-training Model with Prompt-based Learning for Multimodal Emotion Recognition
Multimodal emotion recognition study is hindered by the lack of labelled corpora in terms of scale and diversity, due to the high annotation cost and label ambiguity. In this paper, we propose a pre-training model \textb…
DiversityEmotion ClassificationEmotion RecognitionMultimodal Emotion Recognition+1Scaling Ambiguity: Augmenting Human Annotation in Speech Emotion Recognition with Audio-Language Models
Speech Emotion Recognition models typically use single categorical labels, overlooking the inherent ambiguity of human emotions. Ambiguous Emotion Recognition addresses this by representing emotions as probability distri…
Speech Emotion RecognitionMEDUSA: A Multimodal Deep Fusion Multi-Stage Training Framework for Speech Emotion Recognition in Naturalistic Conditions
SER is a challenging task due to the subjective nature of human emotions and their uneven representation under naturalistic conditions. We propose MEDUSA, a multimodal framework with a four-stage training pipeline, which…
Emotion RecognitionSpeech Emotion RecognitionEnhancing Multimodal Emotion Recognition through Multi-Granularity Cross-Modal Alignment
Multimodal emotion recognition (MER), leveraging speech and text, has emerged as a pivotal domain within human-computer interaction, demanding sophisticated methods for effective multimodal integration. The challenge of …
cross-modal alignmentEmotion RecognitionMultimodal Emotion Recognition