Corpus-informed Retrieval Augmented Generation of Clarifying Questions
This study aims to develop models that generate corpus informed clarifying questions for web search, in a way that ensures the questions align with the available information in the retrieval corpus. We demonstrate the effectiveness of Retrieval Augmented Language Models (RAG) in this process, emphasising their ability to (i) jointly model the user query and retrieval corpus to pinpoint the uncertainty and ask for clarifications end-to-end and (ii) model more evidence documents, which can be used towards increasing the breadth of the questions asked. However, we observe that in current datasets search intents are largely unsupported by the corpus, which is problematic both for training and evaluation. This causes question generation models to ``hallucinate'', ie. suggest intents that are not in the corpus, which can have detrimental effects in performance. To address this, we propose dataset augmentation methods that align the ground truth clarifications with the retrieval corpus. Additionally, we explore techniques to enhance the relevance of the evidence pool during inference, but find that identifying ground truth intents within the corpus remains challenging. Our analysis suggests that this challenge is partly due to the bias of current datasets towards clarification taxonomies and calls for data that can support generating corpus-informed clarifications.
Code (0)
등록된 구현이 없습니다.
Tasks
Question GenerationQuestion-GenerationRAGRetrievalRetrieval-augmented GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Embedding-Informed Adaptive Retrieval-Augmented Generation of Large Language Models
Retrieval-augmented large language models (LLMs) have been remarkably competent in various NLP tasks. However, it was observed by previous works that retrieval is not always helpful, especially when the LLM is already kn…
Continual LearningRetrievalRetrieval-augmented GenerationPAQA: Toward ProActive Open-Retrieval Question Answering
Conversational systems have made significant progress in generating natural language responses. However, their potential as conversational search systems is currently limited due to their passive role in the information-…
Conversational SearchPassage RetrievalQuestion AnsweringRetrievalContext-augmented Retrieval: A Novel Framework for Fast Information Retrieval based Response Generation using Large Language Model
Generating high-quality answers consistently by providing contextual information embedded in the prompt passed to the Large Language Model (LLM) is dependent on the quality of information retrieval. As the corpus of cont…
Answer GenerationInformation RetrievalLanguage ModelingLanguage Modelling+8Mind the Gap: Aligning Knowledge Bases with User Needs to Enhance Mental Health Retrieval
Access to reliable mental health information is vital for early help-seeking, yet expanding knowledge bases is resource-intensive and often misaligned with user needs. This results in poor performance of retrieval system…
From Questions to Insightful Answers: Building an Informed Chatbot for University Resources
This paper presents BARKPLUG V.2, a Large Language Model (LLM)-based chatbot system built using Retrieval Augmented Generation (RAG) pipelines to enhance the user experience and access to information within academic sett…
ChatbotLanguage ModelingLanguage ModellingLarge Language Model+4