Emotion Recognition from Multiple Modalities: Fundamentals and Methodologies
Humans are emotional creatures. Multiple modalities are often involved when we express emotions, whether we do so explicitly (e.g., facial expression, speech) or implicitly (e.g., text, image). Enabling machines to have emotional intelligence, i.e., recognizing, interpreting, processing, and simulating emotions, is becoming increasingly important. In this tutorial, we discuss several key aspects of multi-modal emotion recognition (MER). We begin with a brief introduction on widely used emotion representation models and affective modalities. We then summarize existing emotion annotation strategies and corresponding computational tasks, followed by the description of main challenges in MER. Furthermore, we present some representative approaches on representation learning of each affective modality, feature fusion of different affective modalities, classifier optimization for MER, and domain adaptation for MER. Finally, we outline several real-world applications and discuss some future directions.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationEmotional IntelligenceEmotion RecognitionRepresentation LearningSimilar Papers 제목 키워드 기반
Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalities
Emotion recognition and generation have emerged as crucial topics in Artificial Intelligence research, playing a significant role in enhancing human-computer interaction within healthcare, customer service, and other fie…
Emotion RecognitionImpact of multiple modalities on emotion recognition: investigation into 3d facial landmarks, action units, and physiological data
To fully understand the complexities of human emotion, the integration of multiple physical features from different modalities can be advantageous. Considering this, we present an analysis of 3D facial data, action units…
Emotion RecognitionEmoBed: Strengthening Monomodal Emotion Recognition via Training with Crossmodal Emotion Embeddings
Despite remarkable advances in emotion recognition, they are severely restrained from either the essentially limited property of the employed single modality, or the synchronous presence of all involved multiple modaliti…
Emotion ClassificationEmotion RecognitionSemi-supervised Bayesian Deep Multi-modal Emotion Recognition
In emotion recognition, it is difficult to recognize human's emotional states using just a single modality. Besides, the annotation of physiological emotional data is particularly expensive. These two aspects make the bu…
Emotion RecognitionImputationMultimodal Utterance-level Affect Analysis using Visual, Audio and Text Features
The integration of information across multiple modalities and across time is a promising way to enhance the emotion recognition performance of affective systems. Much previous work has focused on instantaneous emotion re…
Emotion Recognition