Toward Understanding The Effect Of Loss function On Then Performance Of Knowledge Graph Embedding
Knowledge graphs (KGs) represent world's facts in structured forms. KG completion exploits the existing facts in a KG to discover new ones. Translation-based embedding model (TransE) is a prominent formulation to do KG completion. Despite the efficiency of TransE in memory and time, it suffers from several limitations in encoding relation patterns such as symmetric, reflexive etc. To resolve this problem, most of the attempts have circled around the revision of the score function of TransE i.e., proposing a more complicated score function such as Trans(A, D, G, H, R, etc) to mitigate the limitations. In this paper, we tackle this problem from a different perspective. We show that existing theories corresponding to the limitations of TransE are inaccurate because they ignore the effect of loss function. Accordingly, we pose theoretical investigations of the main limitations of TransE in the light of loss function. To the best of our knowledge, this has not been investigated so far comprehensively. We show that by a proper selection of the loss function for training the TransE model, the main limitations of the model are mitigated. This is explained by setting upper-bound for the scores of positive samples, showing the region of truth (i.e., the region that a triple is considered positive by the model). Our theoretical proofs with experimental results fill the gap between the capability of translation-based class of embedding models and the loss function. The theories emphasise the importance of the selection of the loss functions for training the models. Our experimental evaluations on different loss functions used for training the models justify our theoretical proofs and confirm the importance of the loss functions on the performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph EmbeddingKnowledge Graph CompletionKnowledge Graph EmbeddingKnowledge GraphsTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On the implicit minimization of alternative loss functions when training deep networks
Understanding the implicit bias of optimization algorithms is important in order to improve generalization of neural networks. One approach to try to exploit such understanding would be to then make the bias explicit in …
Inductive BiasSemantic Loss Application to Entity Relation Recognition
Usually, entity relation recognition systems either use a pipe-lined model that treats the entity tagging and relation identification as separate tasks or a joint model that simultaneously identifies the relation and ent…
Feature EngineeringNatural Language UnderstandingRelationRelation ExtractionRethinking Parameter Counting: Effective Dimensionality Revisited
Neural networks appear to have mysterious generalization properties when using parameter counting as a proxy for complexity. Indeed, neural networks often have many more parameters than there are data points, yet still p…
DiversityModel SelectionRethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited
Neural networks appear to have mysterious generalization properties when using parameter counting as a proxy for complexity. Indeed, neural networks often have many more parameters than there are data points, yet still p…
DiversityModel SelectionPI-Att: Topology Attention for Segmentation Networks through Adaptive Persistence Image Representation
Segmenting multiple objects (e.g., organs) in medical images often requires an understanding of their topology, which simultaneously quantifies the shape of the objects and their positions relative to each other. This un…
Medical Image AnalysisSegmentation