AprilE: Attention with Pseudo Residual Connection for Knowledge Graph Embedding
Knowledge graph embedding maps entities and relations into low-dimensional vector space. However, it is still challenging for many existing methods to model diverse relational patterns, especially symmetric and antisymmetric relations. To address this issue, we propose a novel model, AprilE, which employs triple-level self-attention and pseudo residual connection to model relational patterns. The triple-level self-attention treats head entity, relation, and tail entity as a sequence and captures the dependency within a triple. At the same time the pseudo residual connection retains primitive semantic features. Furthermore, to deal with symmetric and antisymmetric relations, two schemas of score function are designed via a position-adaptive mechanism. Experimental results on public datasets demonstrate that our model can produce expressive knowledge embedding and significantly outperforms most of the state-of-the-art works.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph EmbeddingKnowledge Graph EmbeddingPositionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
XAttnRes: Cross-Stage Attention Residuals for Medical Image Segmentation
In the field of Large Language Models (LLMs), Attention Residuals have recently demonstrated that learned, selective aggregation over all preceding layer outputs can outperform fixed residual connections. We propose Cros…
Medical Image SegmentationRRA: Recurrent Residual Attention for Sequence Learning
In this paper, we propose a recurrent neural network (RNN) with residual attention (RRA) to learn long-range dependencies from sequential data. We propose to add residual connections across timesteps to RNN, which explic…
Sentiment AnalysisImage Super-Resolution Using Very Deep Residual Channel Attention Networks
Convolutional neural network (CNN) depth is of crucial importance for image super-resolution (SR). However, we observe that deeper networks for image SR are more difficult to train. The low-resolution inputs and features…
Image Super-ResolutionSuper-ResolutionA self-training framework for glaucoma grading in OCT B-scans
In this paper, we present a self-training-based framework for glaucoma grading using OCT B-scans under the presence of domain shift. Particularly, the proposed two-step learning methodology resorts to pseudo-labels gener…
A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution
Deep learning methods have shown outstanding performance in many applications, including single-image super-resolution (SISR). With residual connection architecture, deeply stacked convolutional neural networks provide a…
Image Super-ResolutionSuper-Resolution