Emotion Recognition with Pre-Trained Transformers Using Multimodal Signals
In this paper, we address the problem of multimodal emotion recognition from multiple physiological signals. We demonstrate that a Transformer-based approach is suitable for this task. In addition, we present how such models may be pretrained in a multimodal scenario to improve emotion recognition performances. We evaluate the benefits of using multimodal inputs and pre-training with our approach on a state-ofthe-art dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Emotion RecognitionMultimodal Emotion RecognitionSimilar Papers 제목 키워드 기반
Low Rank Fusion based Transformers for Multimodal Sequences
Our senses individually work in a coordinated fashion to express our emotional intentions. In this work, we experiment with modeling modality-specific sensory signals to attend to our latent multimodal emotional intentio…
Emotion RecognitionEmotion recognition based on multi-modal electrophysiology multi-head attention Contrastive Learning
Emotion recognition is an important research direction in artificial intelligence, helping machines understand and adapt to human emotional states. Multimodal electrophysiological(ME) signals, such as EEG, GSR, respirati…
Contrastive LearningEEGEmotion RecognitionMultimodal Emotion RecognitionMulti-Modal Emotion Recognition by Text, Speech and Video Using Pretrained Transformers
Due to the complex nature of human emotions and the diversity of emotion representation methods in humans, emotion recognition is a challenging field. In this research, three input modalities, namely text, audio (speech)…
DiversityEmotion RecognitionMultimodal Emotion RecognitionSelf-Supervised Learning+1Multimodal Emotion Recognition with Transformer-Based Self Supervised Feature Fusion
Emotion Recognition is a challenging research area given its complex nature, and humans express emotional cues across various modalities such as language, facial expressions, and speech. Representation and fusion of feat…
Emotion RecognitionMultimodal Deep LearningMultimodal Emotion RecognitionMultimodal Sentiment Analysis+2Deep Temporal Modeling and Ensemble Fusion for Multimodal Emotion Recognition from Physiological Signals
Physiological stress and emotion recognition are important for health monitoring and affective computing. In this work, we present a comprehensive evaluation of deep learning models such as Long Short-Term Memory (LSTM),…
Multimodal Emotion Recognition