Semi-IIN: Semi-supervised Intra-inter modal Interaction Learning Network for Multimodal Sentiment Analysis

2024-12-13 · Jinhao Lin, Yifei Wang, Yanwu Xu, Qi Liu

Despite multimodal sentiment analysis being a fertile research ground that merits further investigation, current approaches take up high annotation cost and suffer from label ambiguity, non-amicable to high-quality labeled data acquisition. Furthermore, choosing the right interactions is essential because the significance of intra- or inter-modal interactions can differ among various samples. To this end, we propose Semi-IIN, a Semi-supervised Intra-inter modal Interaction learning Network for multimodal sentiment analysis. Semi-IIN integrates masked attention and gating mechanisms, enabling effective dynamic selection after independently capturing intra- and inter-modal interactive information. Combined with the self-training approach, Semi-IIN fully utilizes the knowledge learned from unlabeled data. Experimental results on two public datasets, MOSI and MOSEI, demonstrate the effectiveness of Semi-IIN, establishing a new state-of-the-art on several metrics. Code is available at https://github.com/flow-ljh/Semi-IIN.

📄 PDF Abstract BibTeX arXiv:2412.09784

Code (1)

flow-ljh/semi-iin 공식 구현 pytorch

Tasks

Multimodal Sentiment Analysis Sentiment Analysis

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

Semi-Supervised Image Captioning Considering Wasserstein Graph Matching

2024-03-26 · Yang Yang

Image captioning can automatically generate captions for the given images, and the key challenge is to learn a mapping function from visual features to natural language features. Existing approaches are mostly supervised…

Data Augmentation Graph Matching Image Captioning Sentence

Multi-Modal Representation Learning via Semi-Supervised Rate Reduction for Generalized Category Discovery

2026-02-23 · Wei He, Xianghan Meng, Zhiyuan Huang, Xianbiao Qi 외 arxiv

Generalized Category Discovery (GCD) aims to identify both known and unknown categories, with only partial labels given for the known categories, posing a challenging open-set recognition problem. State-of-the-art approa…

Representation Learning

Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentation

2021-01-07 · Kang Li, Shujun Wang, Lequan Yu, Pheng-Ann Heng

Annotation scarcity is a long-standing problem in medical image analysis area. To efficiently leverage limited annotations, abundant unlabeled data are additionally exploited in semi-supervised learning, while well-estab…

Cardiac Segmentation Domain Adaptation Medical Image Analysis Semi-supervised Domain Adaptation+1

Dual-Teacher: Integrating Intra-domain and Inter-domain Teachers for Annotation-efficient Cardiac Segmentation

2020-07-13 · Kang Li, Shujun Wang, Lequan Yu, Pheng-Ann Heng

Medical image annotations are prohibitively time-consuming and expensive to obtain. To alleviate annotation scarcity, many approaches have been developed to efficiently utilize extra information, e.g.,semi-supervised lea…

Cardiac Segmentation Domain Adaptation Image Segmentation Knowledge Distillation+4

Multimodal Semi-Supervised Learning for 3D Objects

2021-10-22 · Zhimin Chen, Longlong Jing, Yang Liang, YingLi Tian 외

In recent years, semi-supervised learning has been widely explored and shows excellent data efficiency for 2D data. There is an emerging need to improve data efficiency for 3D tasks due to the scarcity of labeled 3D data…

3D Classification Retrieval