Internal and External Knowledge Interactive Refinement Framework for Knowledge-Intensive Question Answering
Recent works have attempted to integrate external knowledge into LLMs to address the limitations and potential factual errors in LLM-generated content. However, how to retrieve the correct knowledge from the large amount of external knowledge imposes a challenge. To this end, we empirically observe that LLMs have already encoded rich knowledge in their pretrained parameters and utilizing these internal knowledge improves the retrieval of external knowledge when applying them to knowledge-intensive tasks. In this paper, we propose a new internal and external knowledge interactive refinement paradigm dubbed IEKR to utilize internal knowledge in LLM to help retrieve relevant knowledge from the external knowledge base, as well as exploit the external knowledge to refine the hallucination of generated internal knowledge. By simply adding a prompt like 'Tell me something about' to the LLMs, we try to review related explicit knowledge and insert them with the query into the retriever for external retrieval. The external knowledge is utilized to complement the internal knowledge into input of LLM for answers. We conduct experiments on 3 benchmark datasets in knowledge-intensive question answering task with different LLMs and domains, achieving the new state-of-the-art. Further analysis shows the effectiveness of different modules in our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
HallucinationQuestion AnsweringRetrievalSimilar Papers 제목 키워드 기반
KnowPath: Knowledge-enhanced Reasoning via LLM-generated Inference Paths over Knowledge Graphs
Large language models (LLMs) have demonstrated remarkable capabilities in various complex tasks, yet they still suffer from hallucinations. Introducing external knowledge, such as knowledge graph, can enhance the LLMs' a…
Knowledge GraphsSCEF: A Support-Confidence-aware Embedding Framework for Knowledge Graph Refinement
Knowledge graph (KG) refinement mainly aims at KG completion and correction (i.e., error detection). However, most conventional KG embedding models only focus on KG completion with an unreasonable assumption that all fac…
TranslationTowards an Approach based on Knowledge Graph Refinement for Tabular Data to Knowledge Graph Matching
This paper presents our contribution to the Accuracy Track of Semantic Web Challenge on Tabular Data to Knowledge Graph Matching (SemTab). This contribution consists of the proposition of an approach based on knowledge g…
Cell Entity AnnotationColumn Type AnnotationGraph MatchingPositionActiveRAG: Autonomously Knowledge Assimilation and Accommodation through Retrieval-Augmented Agents
Retrieval-Augmented Generation (RAG) enables Large Language Models (LLMs) to leverage external knowledge, enhancing their performance on knowledge-intensive tasks. However, existing RAG models often treat LLMs as passive…
Active LearningPositionQuestion AnsweringRAG+2How Large Language Models Balance Internal Knowledge with User and Document Assertions
Large language models (LLMs) often need to balance their internal parametric knowledge with external information, such as user beliefs and content from retrieved documents, in real-world scenarios like RAG or chat-based …