Evaluating Knowledge Graph Based Retrieval Augmented Generation Methods under Knowledge Incompleteness

2025-04-07 · Dongzhuoran Zhou, Yuqicheng Zhu, Yuan He, Jiaoyan Chen, Evgeny Kharlamov, Steffen Staab

Knowledge Graph based Retrieval-Augmented Generation (KG-RAG) is a technique that enhances Large Language Model (LLM) inference in tasks like Question Answering (QA) by retrieving relevant information from knowledge graphs (KGs). However, real-world KGs are often incomplete, meaning that essential information for answering questions may be missing. Existing benchmarks do not adequately capture the impact of KG incompleteness on KG-RAG performance. In this paper, we systematically evaluate KG-RAG methods under incomplete KGs by removing triples using different methods and analyzing the resulting effects. We demonstrate that KG-RAG methods are sensitive to KG incompleteness, highlighting the need for more robust approaches in realistic settings.

📄 PDF Abstract BibTeX arXiv:2504.05163

Code (0)

등록된 구현이 없습니다.

Tasks

Knowledge GraphsLanguage ModelingLanguage ModellingLarge Language ModelQuestion Answering RAG RetrievalRetrieval-augmented Generation

mmRAG: A Modular Benchmark for Retrieval-Augmented Generation over Text, Tables, and Knowledge Graphs

2025-05-16 · Chuan Xu, Qiaosheng Chen, Yutong Feng, Gong Cheng

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for enhancing the capabilities of large language models. However, existing RAG evaluation predominantly focuses on text retrieval and relies on opaq…

Information Retrieval Knowledge Graphs Question Answering RAG+3

Enhancing Scientific Literature Chatbots with Retrieval-Augmented Generation: A Performance Evaluation of Vector and Graph-Based Systems

2026-02-19 · Hamideh Ghanadian, Amin Kamali, Mohammad Hossein Tekieh arxiv

This paper investigates the enhancement of scientific literature chatbots through retrieval-augmented generation (RAG), with a focus on evaluating vector- and graph-based retrieval systems. The proposed chatbot leverages…

Decision Making

Retrieval Augmented (Knowledge Graph), and Large Language Model-Driven Design Structure Matrix (DSM) Generation of Cyber-Physical Systems

2026-01-30 · H. Sinan Bank, Daniel R. Herber arxiv

We explore the potential of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and Graph-based RAG (GraphRAG) for generating Design Structure Matrices (DSMs). We test these methods on two distinct use ca…

From Flat to Structural: Enhancing Automated Short Answer Grading with GraphRAG

2026-02-28 · Yucheng Chu, Haoyu Han, Shen Dong, Hang Li 외 arxiv

Automated short answer grading (ASAG) is critical for scaling educational assessment, yet large language models (LLMs) often struggle with hallucinations and strict rubric adherence due to their reliance on generalized p…

Logical Reasoning

Evaluating Cultural Knowledge Processing in Large Language Models: A Cognitive Benchmarking Framework Integrating Retrieval-Augmented Generation

2025-11-03 · Hung-Shin Lee, Chen-Chi Chang, Ching-Yuan Chen, Yun-Hsiang Hsu arxiv

This study proposes a cognitive benchmarking framework to evaluate how large language models (LLMs) process and apply culturally specific knowledge. The framework integrates Bloom's Taxonomy with Retrieval-Augmented Gene…