Multi-Modal Sentiment Analysis with Dynamic Attention Fusion
Traditional sentiment analysis has long been a unimodal task, relying solely on text. This approach overlooks non-verbal cues such as vocal tone and prosody that are essential for capturing true emotional intent. We introduce Dynamic Attention Fusion (DAF), a lightweight framework that combines frozen text embeddings from a pretrained language model with acoustic features from a speech encoder, using an adaptive attention mechanism to weight each modality per utterance. Without any finetuning of the underlying encoders, our proposed DAF model consistently outperforms both static fusion and unimodal baselines on a large multimodal benchmark. We report notable gains in F1-score and reductions in prediction error and perform a variety of ablation studies that support our hypothesis that the dynamic weighting strategy is crucial for modeling emotionally complex inputs. By effectively integrating verbal and non-verbal information, our approach offers a more robust foundation for sentiment prediction and carries broader impact for affective computing applications -- from emotion recognition and mental health assessment to more natural human computer interaction.
Code (0)
등록된 구현이 없습니다.
Tasks
Emotion RecognitionSentiment AnalysisSimilar Papers 제목 키워드 기반
Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysis
Multimodal Sentiment Analysis (MSA) utilizes multimodal data to infer the users' sentiment. Previous methods focus on equally treating the contribution of each modality or statically using text as the dominant modality t…
Multimodal Sentiment AnalysisSentiment AnalysisAdaptiSent: Context-Aware Adaptive Attention for Multimodal Aspect-Based Sentiment Analysis
We introduce AdaptiSent, a new framework for Multimodal Aspect-Based Sentiment Analysis (MABSA) that uses adaptive cross-modal attention mechanisms to improve sentiment classification and aspect term extraction from both…
Aspect-Based Sentiment AnalysisSentiment AnalysisSentiment ClassificationTerm ExtractionDynamic Multimodal Sentiment Analysis: Leveraging Cross-Modal Attention for Enabled Classification
This paper explores the development of a multimodal sentiment analysis model that integrates text, audio, and visual data to enhance sentiment classification. The goal is to improve emotion detection by capturing the com…
Multimodal Sentiment AnalysisSentiment AnalysisSentiment ClassificationMulti-channel Attentive Graph Convolutional Network With Sentiment Fusion For Multimodal Sentiment Analysis
Nowadays, with the explosive growth of multimodal reviews on social media platforms, multimodal sentiment analysis has recently gained popularity because of its high relevance to these social media posts. Although most p…
Multimodal Sentiment AnalysisSentiment AnalysisBACN: Bi-direction Attention Capsule-based Network for Multimodal Sentiment Analysis
Capsule-based network has currently identified its effectiveness in analyzing the heterogeneity issue of multimodal sentiment analysis. However, existing manners could only exploit the spatial relation between represent…
Multimodal Sentiment AnalysisSentiment Analysis