Mutilmodal Feature Extraction and Attention-based Fusion for Emotion Estimation in Videos
The continuous improvement of human-computer interaction technology makes it possible to compute emotions. In this paper, we introduce our submission to the CVPR 2023 Competition on Affective Behavior Analysis in-the-wild (ABAW). Sentiment analysis in human-computer interaction should, as far as possible Start with multiple dimensions, fill in the single imperfect emotion channel, and finally determine the emotion tendency by fitting multiple results. Therefore, We exploited multimodal features extracted from video of different lengths from the competition dataset, including audio, pose and images. Well-informed emotion representations drive us to propose a Attention-based multimodal framework for emotion estimation. Our system achieves the performance of 0.361 on the validation dataset. The code is available at [https://github.com/xkwangcn/ABAW-5th-RT-IAI].
Code (1)
Tasks
Sentiment AnalysisSimilar Papers 제목 키워드 기반
Multimodal Emotion Recognition using Audio-Video Transformer Fusion with Cross Attention
Understanding emotions is a fundamental aspect of human communication. Integrating audio and video signals offers a more comprehensive understanding of emotional states compared to traditional methods that rely on a sing…
Emotion RecognitionMultimodal Emotion RecognitionUncertainty-Aware 3D Emotional Talking Face Synthesis with Emotion Prior Distillation
Emotional Talking Face synthesis is pivotal in multimedia and signal processing, yet existing 3D methods suffer from two critical challenges: poor audio-vision emotion alignment, manifested as difficult audio emotion ext…
Enhancing Multimodal Emotion Recognition via Multi-Feature Encoding and Attention-Based Fusion
Multimodal emotion recognition has attracted growing interest due to its importance in human-computer interaction, remote education, and healthcare. This paper proposes a novel multimodal emotion recognition framework th…
Multimodal Emotion RecognitionMilmer: a Framework for Multiple Instance Learning based Multimodal Emotion Recognition
Emotions play a crucial role in human behavior and decision-making, making emotion recognition a key area of interest in human-computer interaction (HCI). This study addresses the challenges of emotion recognition by int…
EEGElectroencephalogram (EEG)Emotion RecognitionMultimodal Emotion Recognition+1Multimodal Emotion-Cause Pair Extraction in Conversations
Emotion cause analysis has received considerable attention in recent years. Previous studies primarily focused on emotion cause extraction from texts in news articles or microblogs. It is also interesting to discover emo…
ArticlesEmotion Cause ExtractionEmotion-Cause Pair ExtractionEmotion Recognition+1