Swift and Sure: Hardness-aware Contrastive Learning for Low-dimensional Knowledge Graph Embeddings
Knowledge graph embedding (KGE) has shown great potential in automatic knowledge graph (KG) completion and knowledge-driven tasks. However, recent KGE models suffer from high training cost and large storage space, thus limiting their practicality in real-world applications. To address this challenge, based on the latest findings in the field of Contrastive Learning, we propose a novel KGE training framework called Hardness-aware Low-dimensional Embedding (HaLE). Instead of the traditional Negative Sampling, we design a new loss function based on query sampling that can balance two important training targets, Alignment and Uniformity. Furthermore, we analyze the hardness-aware ability of recent low-dimensional hyperbolic models and propose a lightweight hardness-aware activation mechanism. The experimental results show that in the limited training time, HaLE can effectively improve the performance and training speed of KGE models on five commonly-used datasets. After training just a few minutes, the HaLE-trained models are competitive compared to the state-of-the-art models in both low- and high-dimensional conditions.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningGraph EmbeddingKnowledge Graph EmbeddingKnowledge Graph EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learning
Self-supervised learning (SSL) has gained remarkable success, for which contrastive learning (CL) plays a key role. However, the recent development of new non-CL frameworks has achieved comparable or better performance w…
Contrastive LearningDiversitySelf-Supervised LearningSwiFT: Swin 4D fMRI Transformer
Modeling spatiotemporal brain dynamics from high-dimensional data, such as functional Magnetic Resonance Imaging (fMRI), is a formidable task in neuroscience. Existing approaches for fMRI analysis utilize hand-crafted fe…
Hard Sample Aware Network for Contrastive Deep Graph Clustering
Contrastive deep graph clustering, which aims to divide nodes into disjoint groups via contrastive mechanisms, is a challenging research spot. Among the recent works, hard sample mining-based algorithms have achieved gre…
AttributeClusteringGraph ClusteringHaSa: Hardness and Structure-Aware Contrastive Knowledge Graph Embedding
We consider a contrastive learning approach to knowledge graph embedding (KGE) via InfoNCE. For KGE, efficient learning relies on augmenting the training data with negative triples. However, most KGE works overlook the b…
Contrastive LearningGraph EmbeddingKnowledge Graph CompletionKnowledge Graph Embedding+2Structure-Aware Hard Negative Mining for Heterogeneous Graph Contrastive Learning
Recently, heterogeneous Graph Neural Networks (GNNs) have become a de facto model for analyzing HGs, while most of them rely on a relative large number of labeled data. In this work, we investigate Contrastive Learning (…
Contrastive Learning