MoCoKGC: Momentum Contrast Entity Encoding for Knowledge Graph Completion
In recent years, numerous studies have sought to enhance the capabilities of pretrained language models (PLMs) for Knowledge Graph Completion (KGC) tasks by integrating structural information from knowledge graphs. However, existing approaches have not effectively combined the structural attributes of knowledge graphs with the textual descriptions of entities to generate robust entity encodings.To address this issue, this paper proposes MoCoKGC (Momentum Contrast Entity Encoding for Knowledge Graph Completion), which incorporates three primary encoders: the entity-relation encoder, the entity encoder, and the momentum entity encoder. Momentum contrastive learning not only provides more negative samples but also allows for the gradual updating of entity encodings. Consequently, we reintroduce the generated entity encodings into the encoder to incorporate the graph’s structural information.Additionally, MoCoKGC enhances the inferential capabilities of the entity-relation encoder through deep prompts of relations. On the standard evaluation metric, Mean Reciprocal Rank (MRR), the MoCoKGC model demonstrates superior performance, achieving a 7.1% improvement on the WN18RR dataset and an 11% improvement on the Wikidata5M dataset, while also surpassing the current best model on the FB15k-237 dataset. Through a series of experiments, this paper thoroughly examines the role and contribution of each component and parameter of the model.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningKnowledge Graph CompletionKnowledge GraphsLink PredictionRelationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MoCoSA: Momentum Contrast for Knowledge Graph Completion with Structure-Augmented Pre-trained Language Models
Knowledge Graph Completion (KGC) aims to conduct reasoning on the facts within knowledge graphs and automatically infer missing links. Existing methods can mainly be categorized into structure-based or description-based.…
Entity EmbeddingsKnowledge Graph CompletionKnowledge GraphsLink PredictionPseudo-Label Calibration Semi-supervised Multi-Modal Entity Alignment
Multi-modal entity alignment (MMEA) aims to identify equivalent entities between two multi-modal knowledge graphs for integration. Unfortunately, prior arts have attempted to improve the interaction and fusion of multi-m…
AttributeContrastive LearningEntity AlignmentKnowledge Graphs+2Momentum Contrastive Pre-training for Question Answering
Existing pre-training methods for extractive Question Answering (QA) generate cloze-like queries different from natural questions in syntax structure, which could overfit pre-trained models to simple keyword matching. In…
BenchmarkingContrastive LearningExtractive Question-AnsweringNatural Questions+1Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning
Federated learning on edge devices must cope with non-IID client data and tight memory budgets. Adaptive optimizers like Adam stabilize training under data heterogeneity but require storing full-precision momentum and va…
Federated LearningAugmented Skeleton Based Contrastive Action Learning with Momentum LSTM for Unsupervised Action Recognition
Action recognition via 3D skeleton data is an emerging important topic in these years. Most existing methods either extract hand-crafted descriptors or learn action representations by supervised learning paradigms that r…
Action RecognitionContrastive LearningSelf-Supervised Human Action RecognitionSelf-supervised Skeleton-based Action Recognition