WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge

2024-01-12 · Wenbin Wang, Liang Ding, Li Shen, Yong Luo, Han Hu, DaCheng Tao

Sentiment analysis is rapidly advancing by utilizing various data modalities (e.g., text, image). However, most previous works relied on superficial information, neglecting the incorporation of contextual world knowledge (e.g., background information derived from but beyond the given image and text pairs) and thereby restricting their ability to achieve better multimodal sentiment analysis (MSA). In this paper, we proposed a plug-in framework named WisdoM, to leverage the contextual world knowledge induced from the large vision-language models (LVLMs) for enhanced MSA. WisdoM utilizes LVLMs to comprehensively analyze both images and corresponding texts, simultaneously generating pertinent context. To reduce the noise in the context, we also introduce a training-free contextual fusion mechanism. Experiments across diverse granularities of MSA tasks consistently demonstrate that our approach has substantial improvements (brings an average +1.96% F1 score among five advanced methods) over several state-of-the-art methods.

📄 PDF Abstract BibTeX arXiv:2401.06659

Code (0)

등록된 구현이 없습니다.

Tasks

Multimodal Sentiment Analysis Sentiment AnalysisWorld Knowledge

Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling

2018-06-16 · N. Majumder, D. Hazarika, A. Gelbukh, E. Cambria 외

Multimodal sentiment analysis is a very actively growing field of research. A promising area of opportunity in this field is to improve the multimodal fusion mechanism. We present a novel feature fusion strategy that pro…

Multimodal Emotion Recognition Multimodal Sentiment Analysis Sentiment Analysis

Event Outcome Prediction using Sentiment Analysis and Crowd Wisdom in Microblog Feeds

2019-12-11 · Rahul Radhakrishnan Iyer, Ronghuo Zheng, Yuezhang Li, Katia Sycara

Sentiment Analysis of microblog feeds has attracted considerable interest in recent times. Most of the current work focuses on tweet sentiment classification. But not much work has been done to explore how reliable the o…

Classification General Classification Multi-Label Classification MUlTI-LABEL-ClASSIFICATION+2

A Novel Context-Aware Multimodal Framework for Persian Sentiment Analysis

2021-03-03 · Kia Dashtipour, Mandar Gogate, Erik Cambria, Amir Hussain

Most recent works on sentiment analysis have exploited the text modality. However, millions of hours of video recordings posted on social media platforms everyday hold vital unstructured information that can be exploited…

Multimodal Sentiment Analysis Persian Sentiment Analysis Sentiment Analysis

Leveraging Textual-Cues for Enhancing Multimodal Sentiment Analysis by Object Recognition

2026-01-30 · Sumana Biswas, Karen Young, Josephine Griffith arxiv

Multimodal sentiment analysis, which includes both image and text data, presents several challenges due to the dissimilarities in the modalities of text and image, the ambiguity of sentiment, and the complexities of cont…

Multimodal Sentiment Analysis Object Recognition

Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Survey

2024-06-12 · Hao Yang, Yanyan Zhao, Yang Wu, Shilong Wang 외

Compared to traditional sentiment analysis, which only considers text, multimodal sentiment analysis needs to consider emotional signals from multimodal sources simultaneously and is therefore more consistent with the wa…

Multimodal Sentiment Analysis Sentiment Analysis