Modality-Invariant Bidirectional Temporal Representation Distillation Network for Missing Multimodal Sentiment Analysis
Multimodal Sentiment Analysis (MSA) integrates diverse modalities(text, audio, and video) to comprehensively analyze and understand individuals' emotional states. However, the real-world prevalence of incomplete data poses significant challenges to MSA, mainly due to the randomness of modality missing. Moreover, the heterogeneity issue in multimodal data has yet to be effectively addressed. To tackle these challenges, we introduce the Modality-Invariant Bidirectional Temporal Representation Distillation Network (MITR-DNet) for Missing Multimodal Sentiment Analysis. MITR-DNet employs a distillation approach, wherein a complete modality teacher model guides a missing modality student model, ensuring robustness in the presence of modality missing. Simultaneously, we developed the Modality-Invariant Bidirectional Temporal Representation Learning Module (MIB-TRL) to mitigate heterogeneity.
Code (0)
등록된 구현이 없습니다.
Tasks
Multimodal Sentiment AnalysisRepresentation LearningSentiment AnalysisSimilar Papers 제목 키워드 기반
Learning Language-Driven Sequence-Level Modal-Invariant Representations for Video-Based Visible-Infrared Person Re-Identification
The core of video-based visible-infrared person re-identification (VVI-ReID) lies in learning sequence-level modal-invariant representations across different modalities. Recent research tends to use modality-shared langu…
Person Re-IdentificationRepresentation LearningDINOv2 Driven Gait Representation Learning for Video-Based Visible-Infrared Person Re-identification
Video-based Visible-Infrared person re-identification (VVI-ReID) aims to retrieve the same pedestrian across visible and infrared modalities from video sequences. Existing methods tend to exploit modality-invariant visua…
Person Re-IdentificationRepresentation LearningCross-Modal RetrievalSemantic-Guided Multimodal Sentiment Decoding with Adversarial Temporal-Invariant Learning
Multimodal sentiment analysis aims to learn representations from different modalities to identify human emotions. However, existing works often neglect the frame-level redundancy inherent in continuous time series, resul…
Multimodal Sentiment AnalysisSentiment AnalysisTime SeriesC2KD: Bridging the Modality Gap for Cross-Modal Knowledge Distillation
Existing Knowledge Distillation (KD) methods typically focus on transferring knowledge from a large-capacity teacher to a low-capacity student model achieving substantial success in unimodal knowledge transfer. Howev…
Knowledge DistillationTransfer LearningRethinking RGB-Event Semantic Segmentation with a Novel Bidirectional Motion-enhanced Event Representation
Event cameras capture motion dynamics, offering a unique modality with great potential in various computer vision tasks. However, RGB-Event fusion faces three intrinsic misalignments: (i) temporal, (ii) spatial, and (iii…
Semantic Segmentation