Multimodal Emotion Recognition and Sentiment Analysis in Multi-Party Conversation Contexts
Emotion recognition and sentiment analysis are pivotal tasks in speech and language processing, particularly in real-world scenarios involving multi-party, conversational data. This paper presents a multimodal approach to tackle these challenges on a well-known dataset. We propose a system that integrates four key modalities/channels using pre-trained models: RoBERTa for text, Wav2Vec2 for speech, a proposed FacialNet for facial expressions, and a CNN+Transformer architecture trained from scratch for video analysis. Feature embeddings from each modality are concatenated to form a multimodal vector, which is then used to predict emotion and sentiment labels. The multimodal system demonstrates superior performance compared to unimodal approaches, achieving an accuracy of 66.36% for emotion recognition and 72.15% for sentiment analysis.
Code (0)
등록된 구현이 없습니다.
Tasks
Emotion RecognitionMultimodal Emotion RecognitionSentiment AnalysisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition
Multimodal sentiment analysis (MSA) and emotion recognition in conversation (ERC) are key research topics for computers to understand human behaviors. From a psychological perspective, emotions are the expression of affe…
Contrastive LearningEmotion RecognitionEmotion Recognition in ConversationMultimodal Sentiment Analysis+1EmoVerse: Exploring Multimodal Large Language Models for Sentiment and Emotion Understanding
Sentiment and emotion understanding are essential to applications such as human-computer interaction and depression detection. While Multimodal Large Language Models (MLLMs) demonstrate robust general capabilities, they …
Depression DetectionEmotion-Cause Pair ExtractionEmotion RecognitionFacial Expression Recognition+3A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysis
Understanding expressed sentiment and emotions are two crucial factors in human multimodal language. This paper describes a Transformer-based joint-encoding (TBJE) for the task of Emotion Recognition and Sentiment Analys…
Emotion RecognitionMultimodal Sentiment AnalysisSentiment AnalysisUniSA: Unified Generative Framework for Sentiment Analysis
Sentiment analysis is a crucial task that aims to understand people's emotional states and predict emotional categories based on multimodal information. It consists of several subtasks, such as emotion recognition in con…
Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)Emotion RecognitionEmotion Recognition in Conversation+2MART: Masked Affective RepresenTation Learning via Masked Temporal Distribution Distillation
Limited training data is a long-standing problem for video emotion analysis (VEA). Existing works leverage the power of large-scale image datasets for transferring while failing to extract the temporal correlation of…
Emotion RecognitionMultimodal Emotion RecognitionMultimodal Sentiment AnalysisRepresentation Learning+2