ICON: Interactive Conversational Memory Network for Multimodal Emotion Detection
Emotion recognition in conversations is crucial for building empathetic machines. Present works in this domain do not explicitly consider the inter-personal influences that thrive in the emotional dynamics of dialogues. To this end, we propose Interactive COnversational memory Network (ICON), a multimodal emotion detection framework that extracts multimodal features from conversational videos and hierarchically models the self- and inter-speaker emotional influences into global memories. Such memories generate contextual summaries which aid in predicting the emotional orientation of utterance-videos. Our model outperforms state-of-the-art networks on multiple classification and regression tasks in two benchmark datasets.
Code (1)
Tasks
Emotion RecognitionEmotion Recognition in ConversationGeneral ClassificationMultimodal Emotion RecognitionregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Contextualized Real-Time Multimodal Emotion Recognition for Conversational Agents using Graph Convolutional Networks in Reinforcement Learning
Owing to the recent developments in Generative Artificial Intelligence (GenAI) and Large Language Models (LLM), conversational agents are becoming increasingly popular and accepted. They provide a human touch by interact…
Emotion ClassificationEmotion RecognitionMultimodal Emotion Recognitionreinforcement-learning+2Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos
Emotion recognition in conversations is crucial for the development of empathetic machines. Present methods mostly ignore the role of inter-speaker dependency relations while classifying emotions in conversations. In thi…
Emotion RecognitionEmotion Recognition in ConversationCOIN: Conversational Interactive Networks for Emotion Recognition in Conversation
Emotion recognition in conversation has received considerable attention recently because of its practical industrial applications. Existing methods tend to overlook the immediate mutual interaction between different spea…
Emotion RecognitionEmotion Recognition in ConversationEmpathy Through Multimodality in Conversational Interfaces
Agents represent one of the most emerging applications of Large Language Models (LLMs) and Generative AI, with their effectiveness hinging on multimodal capabilities to navigate complex user environments. Conversational …
Emotional IntelligenceEmotion RecognitionMultimodal Emotion RecognitionNavigateEvaluating multimodal emotion recognition in proactive conversational agents: A user study
This article presents a multimodal emotion recognition module integrated into a proactive Socially Interactive Agent (SIA) powered by generative artificial intelligence. The system evaluates real-time affective states th…
Multimodal Emotion Recognition