Knowledge Representation Learning with Contrastive Completion Coding
Knowledge representation learning (KRL) has been used in plenty of knowledge-driven tasks. Despite fruitfully progress, existing methods still suffer from the immaturity on tackling potentially-imperfect knowledge graphs and highly-imbalanced positive-negative instances during training, both of which would hinder the performance of KRL. In this paper, we propose Contrastive Completion Coding (C^3), a novel KRL framework that is composed of two functional components: 1. Hierarchical Architecture, which integrates both low-level standalone features and high-level topology-aware features to yield robust embedding for each entity/relation. 2. Normalized Contrasitive Training, which conducts normalized one-to-many contrasitive learning to emphasize different negatives with different weights, delivering better convergence compared to conventional training losses. Extensive experiments on several benchmarks verify the efficacy of the two proposed techniques and combing them together generally achieves superior performance against state-of-the-art approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge GraphsRepresentation LearningSimilar Papers 제목 키워드 기반
Commonsense Knowledge Graph Completion Via Contrastive Pretraining and Node Clustering
The nodes in the commonsense knowledge graph (CSKG) are normally represented by free-form short text (e.g., word or phrase). Different nodes may represent the same concept. This leads to the problems of edge sparsity and…
ClusteringContrastive LearningGraph Representation LearningKnowledge Graph Completion+2MICO: A Multi-alternative Contrastive Learning Framework for Commonsense Knowledge Representation
Commonsense reasoning tasks such as commonsense knowledge graph completion and commonsense question answering require powerful representation learning. In this paper, we propose to learn commonsense knowledge representat…
Contrastive LearningKnowledge Graph CompletionKnowledge GraphsQuestion Answering+2MoCoKGC: Momentum Contrast Entity Encoding for Knowledge Graph Completion
In recent years, numerous studies have sought to enhance the capabilities of pretrained language models (PLMs) for Knowledge Graph Completion (KGC) tasks by integrating structural information from knowledge graphs. Howev…
Contrastive LearningKnowledge Graph CompletionKnowledge GraphsLink Prediction+1Subgraph-Aware Training of Language Models for Knowledge Graph Completion Using Structure-Aware Contrastive Learning
Fine-tuning pre-trained language models (PLMs) has recently shown a potential to improve knowledge graph completion (KGC). However, most PLM-based methods focus solely on encoding textual information, neglecting the long…
Contrastive LearningInductive BiasKnowledge Graph CompletionKnowledge GraphsHICD: Hallucination-Inducing via Attention Dispersion for Contrastive Decoding to Mitigate Hallucinations in Large Language Models
Large Language Models (LLMs) often generate hallucinations, producing outputs that are contextually inaccurate or factually incorrect. We introduce HICD, a novel method designed to induce hallucinations for contrastive d…
HallucinationQuestion AnsweringReading Comprehension