Assessing LLMs Suitability for Knowledge Graph Completion
Recent work has shown the capability of Large Language Models (LLMs) to solve tasks related to Knowledge Graphs, such as Knowledge Graph Completion, even in Zero- or Few-Shot paradigms. However, they are known to hallucinate answers, or output results in a non-deterministic manner, thus leading to wrongly reasoned responses, even if they satisfy the user's demands. To highlight opportunities and challenges in knowledge graphs-related tasks, we experiment with three distinguished LLMs, namely Mixtral-8x7b-Instruct-v0.1, GPT-3.5-Turbo-0125 and GPT-4o, on Knowledge Graph Completion for static knowledge graphs, using prompts constructed following the TELeR taxonomy, in Zero- and One-Shot contexts, on a Task-Oriented Dialogue system use case. When evaluated using both strict and flexible metrics measurement manners, our results show that LLMs could be fit for such a task if prompts encapsulate sufficient information and relevant examples.
Code (1)
Tasks
Knowledge Graph CompletionKnowledge GraphsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On the Use of Entity Embeddings from Pre-Trained Language Models for Knowledge Graph Completion
Recent work has found that entity representations can be extracted from pre-trained language models to develop knowledge graph completion models that are more robust to the naturally occurring sparsity found in knowledge…
Entity EmbeddingsKnowledge Graph CompletionKnowledge GraphsLanguage Modeling+2Cyber Knowledge Completion Using Large Language Models
The integration of the Internet of Things (IoT) into Cyber-Physical Systems (CPSs) has expanded their cyber-attack surface, introducing new and sophisticated threats with potential to exploit emerging vulnerabilities. As…
Binary ClassificationKnowledge Graph CompletionRAGRetrieval-augmented GenerationKaLM: Knowledge-aligned Autoregressive Language Modeling via Dual-view Knowledge Graph Contrastive Learning
Autoregressive large language models (LLMs) pre-trained by next token prediction are inherently proficient in generative tasks. However, their performance on knowledge-driven tasks such as factual knowledge querying rema…
Contrastive LearningGraph Question AnsweringKnowledge Graph CompletionKnowledge Graphs+3KG-CF: Knowledge Graph Completion with Context Filtering under the Guidance of Large Language Models
Large Language Models (LLMs) have shown impressive performance in various tasks, including knowledge graph completion (KGC). However, current studies mostly apply LLMs to classification tasks, like identifying missing tr…
Knowledge Graph CompletionThe Fools are Certain; the Wise are Doubtful: Exploring LLM Confidence in Code Completion
Code completion entails the task of providing missing tokens given a surrounding context. It can boost developer productivity and serve as a code discovery tool. Code completion has recently been approached with Large La…
Code Completion