COGMEN: COntextualized GNN based Multimodal Emotion recognitioN
Emotions are an inherent part of human interactions, and consequently, it is imperative to develop AI systems that understand and recognize human emotions. During a conversation involving various people, a person's emotions are influenced by the other speaker's utterances and their own emotional state over the utterances. In this paper, we propose COntextualized Graph Neural Network based Multimodal Emotion recognitioN (COGMEN) system that leverages local information (i.e., inter/intra dependency between speakers) and global information (context). The proposed model uses Graph Neural Network (GNN) based architecture to model the complex dependencies (local and global information) in a conversation. Our model gives state-of-the-art (SOTA) results on IEMOCAP and MOSEI datasets, and detailed ablation experiments show the importance of modeling information at both levels.
Code (2)
Tasks
Emotion RecognitionEmotion Recognition in ConversationGraph Neural NetworkMultimodal Emotion RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Contextualized Real-Time Multimodal Emotion Recognition for Conversational Agents using Graph Convolutional Networks in Reinforcement Learning
Owing to the recent developments in Generative Artificial Intelligence (GenAI) and Large Language Models (LLM), conversational agents are becoming increasingly popular and accepted. They provide a human touch by interact…
Emotion ClassificationEmotion RecognitionMultimodal Emotion Recognitionreinforcement-learning+2Multimodal Embeddings from Language Models
Word embeddings such as ELMo have recently been shown to model word semantics with greater efficacy through contextualized learning on large-scale language corpora, resulting in significant improvement in state of the ar…
Emotion RecognitionLanguage ModelingLanguage ModellingWord EmbeddingsDouble Multi-Head Attention Multimodal System for Odyssey 2024 Speech Emotion Recognition Challenge
As computer-based applications are becoming more integrated into our daily lives, the importance of Speech Emotion Recognition (SER) has increased significantly. Promoting research with innovative approaches in SER, the …
Emotion RecognitionPositionSpeech Emotion RecognitionCARER: Contextualized Affect Representations for Emotion Recognition
Emotions are expressed in nuanced ways, which varies by collective or individual experiences, knowledge, and beliefs. Therefore, to understand emotion, as conveyed through text, a robust mechanism capable of capturing an…
Emotion RecognitionSemantic Textual SimilarityWord EmbeddingsIn-Depth Analysis of Emotion Recognition through Knowledge-Based Large Language Models
Emotion recognition in social situations is a complex task that requires integrating information from both facial expressions and the situational context. While traditional approaches to automatic emotion recognition hav…
Emotion Recognition