On the Use of Entity Embeddings from Pre-Trained Language Models for Knowledge Graph Completion
Recent work has found that entity representations can be extracted from pre-trained language models to develop knowledge graph completion models that are more robust to the naturally occurring sparsity found in knowledge graphs. In this work, we explore how to best extract and incorporate those embeddings. We explore the suitability of the extracted embeddings for direct use in entity ranking and introduce both unsupervised and supervised processing methods that can lead to improved downstream performance. We then introduce supervised embedding extraction methods and demonstrate that we can extract more informative representations. We also examine the effect of language model selection and find that the choice of model can have a significant impact. We then synthesize our findings and develop a knowledge graph completion model that significantly outperforms recent neural models.
Code (0)
등록된 구현이 없습니다.
Tasks
Entity EmbeddingsKnowledge Graph CompletionKnowledge GraphsLanguage ModelingLanguage ModellingModel SelectionSimilar Papers 제목 키워드 기반
Pretrain-KGEs: Learning Knowledge Representation from Pretrained Models for Knowledge Graph Embeddings
Learning knowledge graph embeddings (KGEs) is an efficient approach to knowledge graph completion. Conventional KGEs often suffer from limited knowledge representation, which causes less accuracy especially when training…
Knowledge Graph CompletionKnowledge Graph EmbeddingsKnowledge GraphsLink Prediction+2CoLAKE: Contextualized Language and Knowledge Embedding
With the emerging branch of incorporating factual knowledge into pre-trained language models such as BERT, most existing models consider shallow, static, and separately pre-trained entity embeddings, which limits the per…
Entity EmbeddingsKnowledge Graph CompletionKnowledge ProbingDBLPLink: An Entity Linker for the DBLP Scholarly Knowledge Graph
In this work, we present a web application named DBLPLink, which performs entity linking over the DBLP scholarly knowledge graph. DBLPLink uses text-to-text pre-trained language models, such as T5, to produce entity labe…
Entity EmbeddingsEntity LinkingEntity Context Graph: Learning Entity Representations fromSemi-Structured Textual Sources on the Web
Knowledge is captured in the form of entities and their relationships and stored in knowledge graphs. Knowledge graphs enhance the capabilities of applications in many different areas including Web search, recommendation…
Entity EmbeddingsGraph LearningKnowledge GraphsLanguage Modelling+1LambdaKG: A Library for Pre-trained Language Model-Based Knowledge Graph Embeddings
Knowledge Graphs (KGs) often have two characteristics: heterogeneous graph structure and text-rich entity/relation information. Text-based KG embeddings can represent entities by encoding descriptions with pre-trained la…
Graph Representation LearningKnowledge Graph CompletionKnowledge Graph EmbeddingsKnowledge Graphs+5