MEKER: Memory Efficient Knowledge Embedding Representation for Link Prediction and Question Answering
Knowledge Graphs (KGs) are symbolically structured storages of facts. The KG embedding contains concise data used in NLP tasks requiring implicit information about the real world. Furthermore, the size of KGs that may be useful in actual NLP assignments is enormous, and creating embedding over it has memory cost issues. We represent KG as a 3rd-order binary tensor and move beyond the standard CP decomposition by using a data-specific generalized version of it. The generalization of the standard CP-ALS algorithm allows obtaining optimization gradients without a backpropagation mechanism. It reduces the memory needed in training while providing computational benefits. We propose a MEKER, a memory-efficient KG embedding model, which yields SOTA-comparable performance on link prediction tasks and KG-based Question Answering.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge GraphsLink PredictionQuestion AnsweringSimilar Papers 제목 키워드 기반
COSINE: Compressive Network Embedding on Large-scale Information Networks
There is recently a surge in approaches that learn low-dimensional embeddings of nodes in networks. As there are many large-scale real-world networks, it's inefficient for existing approaches to store amounts of paramete…
General Classificationgraph partitioningLink PredictionMulti-Label Classification+3Holographic Embeddings of Knowledge Graphs
Learning embeddings of entities and relations is an efficient and versatile method to perform machine learning on relational data such as knowledge graphs. In this work, we propose holographic embeddings (HolE) to learn …
Knowledge GraphsLink PredictionRelational ReasoningLearning Word Representations with Cross-Sentence Dependency for End-to-End Co-reference Resolution
In this work, we present a word embedding model that learns cross-sentence dependency for improving end-to-end co-reference resolution (E2E-CR). While the traditional E2E-CR model generates word representations by runnin…
Coreference ResolutionSentenceHow the (Tensor-) Brain uses Embeddings and Embodiment to Encode Senses and Symbols
The Tensor Brain (TB) has been introduced as a computational model for perception and memory. This paper provides an overview of the TB model, incorporating recent developments and insights into its functionality. The TB…
Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings
Previous knowledge graph embedding approaches usually map entities to representations and utilize score functions to predict the target entities, yet they typically struggle to reason rare or emerging unseen entities. In…
Graph EmbeddingKnowledge Graph EmbeddingKnowledge Graph EmbeddingsLink Prediction+1