MEAformer: Multi-modal Entity Alignment Transformer for Meta Modality Hybrid
Multi-modal entity alignment (MMEA) aims to discover identical entities across different knowledge graphs (KGs) whose entities are associated with relevant images. However, current MMEA algorithms rely on KG-level modality fusion strategies for multi-modal entity representation, which ignores the variations of modality preferences of different entities, thus compromising robustness against noise in modalities such as blurry images and relations. This paper introduces MEAformer, a multi-modal entity alignment transformer approach for meta modality hybrid, which dynamically predicts the mutual correlation coefficients among modalities for more fine-grained entity-level modality fusion and alignment. Experimental results demonstrate that our model not only achieves SOTA performance in multiple training scenarios, including supervised, unsupervised, iterative, and low-resource settings, but also has a limited number of parameters, efficient runtime, and interpretability. Our code is available at https://github.com/zjukg/MEAformer.
Code (1)
Tasks
Entity AlignmentKnowledge GraphsMulti-modal Entity AlignmentMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Accurate and Robust Lesion RECIST Diameter Prediction and Segmentation with Transformers
Automatically measuring lesion/tumor size with RECIST (Response Evaluation Criteria In Solid Tumors) diameters and segmentation is important for computer-aided diagnosis. Although it has been studied in recent years, the…
Lesion SegmentationPredictionregressionSegmentationMulti-Modal Knowledge Graph Transformer Framework for Multi-Modal Entity Alignment
Multi-Modal Entity Alignment (MMEA) is a critical task that aims to identify equivalent entity pairs across multi-modal knowledge graphs (MMKGs). However, this task faces challenges due to the presence of different types…
Entity AlignmentKnowledge GraphsMulti-modal Entity AlignmentMulti-modal Knowledge GraphMyGram: Modality-aware Graph Transformer with Global Distribution for Multi-modal Entity Alignment
Multi-modal entity alignment aims to identify equivalent entities between two multi-modal Knowledge graphs by integrating multi-modal data, such as images and text, to enrich the semantic representations of entities. How…
Multi-modal Entity AlignmentKnowledge GraphsAlign and Prompt: Video-and-Language Pre-training with Entity Prompts
Video-and-language pre-training has shown promising improvements on various downstream tasks. Most previous methods capture cross-modal interactions with a transformer-based multimodal encoder, not fully addressing the m…
cross-modal alignmentEntity AlignmentRetrievalVideo Retrieval+2UniTranSeR: A Unified Transformer Semantic Representation Framework for Multimodal Task-Oriented Dialog System
As a more natural and intelligent interaction manner, multimodal task-oriented dialog system recently has received great attention and many remarkable progresses have been achieved. Nevertheless, almost all existing stud…
Entity Alignment