Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks
Retrieval-augmented generation models have shown state-of-the-art performance across many knowledge-intensive NLP tasks such as open question answering and fact verification. These models are trained to generate a final output given retrieved passages that can be irrelevant to an input query, leading to learning spurious cues or memorization. This work introduces a method to incorporate the evidentiality of passages---whether a passage contains correct evidence to support the output---into training the generator. We introduce a multi-task learning framework to jointly generate the final output and predict the {\it evidentiality} of each passage. We introduce a new task-agnostic method for obtaining high-quality silver evidentiality labels, addressing the issues of gold evidentiality labels being unavailable in most domains. Our experiments on five datasets across three knowledge-intensive tasks of open-domain question answering, fact verification, and knowledge-enhanced dialogue show that our new evidentiality-guided generator significantly outperforms its direct counterpart on all of them, and advances the state of the art on three of them. Our analysis shows that multi-task learning and silver evidentiality mining played key roles.
Code (0)
등록된 구현이 없습니다.
Tasks
Fact VerificationMemorizationMulti-Task LearningOpen-Domain Question AnsweringOpen-Ended Question AnsweringQuestion AnsweringRetrievalRetrieval-augmented GenerationSimilar Papers 제목 키워드 기반
Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks
Retrieval-augmented generation models have shown state-of-the-art performance across many knowledge-intensive NLP tasks such as open question answering and fact verification. These models are trained to generate the fina…
AttributeFact VerificationMemorizationMulti-Task Learning+4ECoRAG: Evidentiality-guided Compression for Long Context RAG
Large Language Models (LLMs) have shown remarkable performance in Open-Domain Question Answering (ODQA) by leveraging external documents through Retrieval-Augmented Generation (RAG). To reduce RAG overhead, from longer c…
Answer GenerationOpen-Domain Question AnsweringQuestion AnsweringRAG+1Robustifying Multi-hop QA through Pseudo-Evidentiality Training
This paper studies the bias problem of multi-hop question answering models, of answering correctly without correct reasoning. One way to robustify these models is by supervising to not only answer right, but also with ri…
counterfactualMulti-hop Question AnsweringQuestion AnsweringWhen Evidence Shapes Collaboration: Knowledge-Conditioned Topology Generation for Multi-Agent Systems
Multi-Agent Systems (MAS) have recently moved from static workflows toward dynamically generated collaboration topologies. However, existing topology generation methods rely primarily on the parametric knowledge of large…
Graph GenerationEvidentiality-aware Retrieval for Overcoming Abstractiveness in Open-Domain Question Answering
The long-standing goal of dense retrievers in abtractive open-domain question answering (ODQA) tasks is to learn to capture evidence passages among relevant passages for any given query, such that the reader produce fact…
Contrastive LearningcounterfactualOpen-Domain Question AnsweringPassage Retrieval+3