Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning
Integrating Large Language Models (LLMs) with Knowledge Graphs (KGs) results in complex systems with numerous hyperparameters that directly affect performance. While such systems are increasingly common in retrieval-augmented generation, the role of systematic hyperparameter optimization remains underexplored. In this paper, we study this problem in the context of Cognee, a modular framework for end-to-end KG construction and retrieval. Using three multi-hop QA benchmarks (HotPotQA, TwoWikiMultiHop, and MuSiQue) we optimize parameters related to chunking, graph construction, retrieval, and prompting. Each configuration is scored using established metrics (exact match, F1, and DeepEval's LLM-based correctness metric). Our results demonstrate that meaningful gains can be achieved through targeted tuning. While the gains are consistent, they are not uniform, with performance varying across datasets and metrics. This variability highlights both the value of tuning and the limitations of standard evaluation measures. While demonstrating the immediate potential of hyperparameter tuning, we argue that future progress will depend not only on architectural advances but also on clearer frameworks for optimization and evaluation in complex, modular systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Chunkinggraph constructionHyperparameter OptimizationKnowledge GraphsRetrievalRetrieval-augmented GenerationSimilar Papers 제목 키워드 기반
Knowledge Management for Automobile Failure Analysis Using Graph RAG
This paper presents a knowledge management system for automobile failure analysis using retrieval-augmented generation (RAG) with large language models (LLMs) and knowledge graphs (KGs). In the automotive industry, there…
Knowledge GraphsManagementRAGRetrieval-augmented Generation+1From Data to Dialogue: Leveraging the Structure of Knowledge Graphs for Conversational Exploratory Search
Exploratory search is an open-ended information retrieval process that aims at discovering knowledge about a topic or domain rather than searching for a specific answer or piece of information. Conversational interfaces …
ArticlesConversational SearchInformation RetrievalKnowledge Graphs+2LLMs+Graphs: Toward Graph-Native, Synergistic AI Systems
Large Language Models (LLMs) have advanced rapidly, but their limitations in structured and multi-hop reasoning underscore the need for graph-native, synergistic artificial intelligence (AI) systems. Graph-structured dat…
Graph Neural NetworkKnowledge GraphsDecision MakingUltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG
Large language models (LLMs) frequently generate confident yet factually incorrect content when used for language generation (a phenomenon often known as hallucination). Retrieval augmented generation (RAG) tries to redu…
Graph Question AnsweringKnowledge GraphsKG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs
Knowledge graphs have emerged as a popular method for injecting up-to-date, factual knowledge into large language models (LLMs). This is typically achieved by converting the knowledge graph into text that the LLM can pro…
Knowledge Graphs