ZeroKBC: A Comprehensive Benchmark for Zero-Shot Knowledge Base Completion
Knowledge base completion (KBC) aims to predict the missing links in knowledge graphs. Previous KBC tasks and approaches mainly focus on the setting where all test entities and relations have appeared in the training set. However, there has been limited research on the zero-shot KBC settings, where we need to deal with unseen entities and relations that emerge in a constantly growing knowledge base. In this work, we systematically examine different possible scenarios of zero-shot KBC and develop a comprehensive benchmark, ZeroKBC, that covers these scenarios with diverse types of knowledge sources. Our systematic analysis reveals several missing yet important zero-shot KBC settings. Experimental results show that canonical and state-of-the-art KBC systems cannot achieve satisfactory performance on this challenging benchmark. By analyzing the strength and weaknesses of these systems on solving ZeroKBC, we further present several important observations and promising future directions.
Code (1)
Tasks
Knowledge Base CompletionKnowledge GraphsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Benchmarking Knowledge-driven Zero-shot Learning
External knowledge (a.k.a. side information) plays a critical role in zero-shot learning (ZSL) which aims to predict with unseen classes that have never appeared in training data. Several kinds of external knowledge, suc…
AttributeBenchmarkingimage-classificationImage Classification+3ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories
Recently, Large Language Models (LLMs) have been serving as general-purpose interfaces, posing a significant demand for comprehensive visual knowledge. However, it remains unclear how well current LLMs and their visually…
Common Sense ReasoningAbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph
Cognitive research indicates that abstraction ability is essential in human intelligence, which remains under-explored in language models. In this paper, we present AbsPyramid, a unified entailment graph of 221K textual …
BenchmarkingSIGN: Spatial-information Incorporated Generative Network for Generalized Zero-shot Semantic Segmentation
Unlike conventional zero-shot classification, zero-shot semantic segmentation predicts a class label at the pixel level instead of the image level. When solving zero-shot semantic segmentation problems, the need for pixe…
Knowledge DistillationSegmentationSemantic Segmentationzero-shot-classification+2FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models
Large language models have demonstrated outstanding performance in various natural language processing tasks, but their security capabilities in the financial domain have not been explored, and their performance on compl…
Multiple-choice