Improving Multimodal Sentiment Analysis: Supervised Angular Margin-based Contrastive Learning for Enhanced Fusion Representation
The effectiveness of a model is heavily reliant on the quality of the fusion representation of multiple modalities in multimodal sentiment analysis. Moreover, each modality is extracted from raw input and integrated with the rest to construct a multimodal representation. Although previous methods have proposed multimodal representations and achieved promising results, most of them focus on forming positive and negative pairs, neglecting the variation in sentiment scores within the same class. Additionally, they fail to capture the significance of unimodal representations in the fusion vector. To address these limitations, we introduce a framework called Supervised Angular-based Contrastive Learning for Multimodal Sentiment Analysis. This framework aims to enhance discrimination and generalizability of the multimodal representation and overcome biases in the fusion vector's modality. Our experimental results, along with visualizations on two widely used datasets, demonstrate the effectiveness of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningMultimodal Sentiment AnalysisSentiment AnalysisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
KDMCSE: Knowledge Distillation Multimodal Sentence Embeddings with Adaptive Angular margin Contrastive Learning
Previous work on multimodal sentence embedding has proposed multimodal contrastive learning and achieved promising results. However, by taking the rest of the batch as negative samples without reviewing when forming cont…
Contrastive LearningKnowledge DistillationSemantic Textual SimilaritySentence+4Variational Fusion for Multimodal Sentiment Analysis
Multimodal fusion is considered a key step in multimodal tasks such as sentiment analysis, emotion detection, question answering, and others. Most of the recent work on multimodal fusion does not guarantee the fidelity o…
Multimodal Sentiment AnalysisQuestion AnsweringSentiment AnalysisCorMulT: A Semi-supervised Modality Correlation-aware Multimodal Transformer for Sentiment Analysis
Multimodal sentiment analysis is an active research area that combines multiple data modalities, e.g., text, image and audio, to analyze human emotions and benefits a variety of applications. Existing multimodal sentimen…
Contrastive LearningMultimodal Sentiment AnalysisPredictionSentiment AnalysisSeq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
Multimodal machine learning is a core research area spanning the language, visual and acoustic modalities. The central challenge in multimodal learning involves learning representations that can process and relate inform…
Multimodal Sentiment AnalysisSentiment AnalysisTranslationWeakly-supervised Multi-task Learning for Multimodal Affect Recognition
Multimodal affect recognition constitutes an important aspect for enhancing interpersonal relationships in human-computer interaction. However, relevant data is hard to come by and notably costly to annotate, which poses…
Emotion RecognitionMulti-Task LearningSentiment Analysis