A Simple Attention-Based Mechanism for Bimodal Emotion Classification
Big data contain rich information for machine learning algorithms to utilize when learning important features during classification tasks. Human beings express their emotion using certain words, speech (tone, pitch, speed) or facial expression. Artificial Intelligence approach to emotion classification are largely based on learning from textual information. However, public datasets containing text and speech data provide sufficient resources to train machine learning algorithms for the tack of emotion classification. In this paper, we present novel bimodal deep learning-based architectures enhanced with attention mechanism trained and tested on text and speech data for emotion classification. We report details of different deep learning based architectures and show the performance of each architecture including rigorous error analyses. Our finding suggests that deep learning based architectures trained on different types of data (text and speech) outperform architectures trained only on text or speech. Our proposed attention-based bimodal architecture outperforms several state-of-the-art systems in emotion classification.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationDeep LearningEmotion ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bimodal Connection Attention Fusion for Speech Emotion Recognition
Multi-modal emotion recognition is challenging due to the difficulty of extracting features that capture subtle emotional differences. Understanding multi-modal interactions and connections is key to building effective b…
DecoderEmotion RecognitionSpeech Emotion RecognitionHeterogeneous bimodal attention fusion for speech emotion recognition
Multi-modal emotion recognition in conversations is a challenging problem due to the complex and complementary interactions between different modalities. Audio and textual cues are particularly important for understandin…
Contrastive LearningEmotion RecognitionSpeech Emotion RecognitionMERGE -- A Bimodal Audio-Lyrics Dataset for Static Music Emotion Recognition
The Music Emotion Recognition (MER) field has seen steady developments in recent years, with contributions from feature engineering, machine learning, and deep learning. The landscape has also shifted from audio-centric …
BenchmarkingDeep LearningEmotion RecognitionFeature Engineering+1IITK at SemEval-2020 Task 8: Unimodal and Bimodal Sentiment Analysis of Internet Memes
Social media is abundant in visual and textual information presented together or in isolation. Memes are the most popular form, belonging to the former class. In this paper, we present our approaches for the Memotion Ana…
Sentiment AnalysisSentiment ClassificationLightweight Models for Multimodal Sequential Data
Human language encompasses more than just text; it also conveys emotions through tone and gestures. We present a case study of three simple and efficient Transformer-based architectures for predicting sentiment and emoti…
Multimodal Sentiment AnalysisSentiment Analysis