Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models
Conventional knowledge graph embedding (KGE) often suffers from limited knowledge representation, leading to performance degradation especially on the low-resource problem. To remedy this, we propose to enrich knowledge representation via pretrained language models by leveraging world knowledge from pretrained models. Specifically, we present a universal training framework named \textit{Pretrain-KGE} consisting of three phases: semantic-based fine-tuning phase, knowledge extracting phase and KGE training phase. Extensive experiments show that our proposed Pretrain-KGE can improve results over KGE models, especially on solving the low-resource problem.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph EmbeddingKnowledge Graph EmbeddingWorld KnowledgeSimilar Papers 제목 키워드 기반
Pretrain-KGEs: Learning Knowledge Representation from Pretrained Models for Knowledge Graph Embeddings
Learning knowledge graph embeddings (KGEs) is an efficient approach to knowledge graph completion. Conventional KGEs often suffer from limited knowledge representation, which causes less accuracy especially when training…
Knowledge Graph CompletionKnowledge Graph EmbeddingsKnowledge GraphsLink Prediction+2mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models
Recent studies have shown that multilingual pretrained language models can be effectively improved with cross-lingual alignment information from Wikipedia entities. However, existing methods only exploit entity informati…
Cross-Lingual Question AnsweringCross-Lingual TransferLanguage ModelingLanguage ModellingMultilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints
Multilingual Knowledge Graph Completion (mKGC) aim at solving queries like (h, r, ?) in different languages by reasoning a tail entity t thus improving multilingual knowledge graphs. Previous studies leverage multilingua…
Knowledge Graph CompletionKnowledge GraphsAutoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?
The success of deep learning heavily relies on large-scale data with comprehensive labels, which is more expensive and time-consuming to fetch in 3D compared to 2D images or natural languages. This promotes the potential…
3D Point Cloud ClassificationFew-Shot 3D Point Cloud ClassificationKnowledge DistillationRepresentation LearningKLMo: Knowledge Graph Enhanced Pretrained Language Model with Fine-Grained Relationships
Interactions between entities in knowledge graph (KG) provide rich knowledge for language representation learning. However, existing knowledge-enhanced pretrained language models (PLMs) only focus on entity information a…
Entity LinkingEntity TypingLanguage ModelingLanguage Modelling+4