Identifying and Answering Questions with False Assumptions: An Interpretable Approach
People often ask questions with false assumptions, a type of question that does not have regular answers. Answering such questions requires first identifying the false assumptions. Large Language Models (LLMs) often generate misleading answers to these questions because of hallucinations. In this paper, we focus on identifying and answering questions with false assumptions in several domains. We first investigate whether the problem reduces to fact verification. Then, we present an approach leveraging external evidence to mitigate hallucinations. Experiments with five LLMs demonstrate that (1) incorporating retrieved evidence is beneficial and (2) generating and validating atomic assumptions yields more improvements and provides an interpretable answer by pinpointing the false assumptions.
Code (0)
등록된 구현이 없습니다.
Tasks
Fact VerificationSimilar Papers 제목 키워드 기반
(QA)$^2$: Question Answering with Questionable Assumptions
Naturally occurring information-seeking questions often contain questionable assumptions -- assumptions that are false or unverifiable. Questions containing questionable assumptions are challenging because they require a…
Question AnsweringSyn-QA2: Evaluating False Assumptions in Long-tail Questions with Synthetic QA Datasets
Sensitivity to false assumptions (or false premises) in information-seeking questions is critical for robust question-answering (QA) systems. Recent work has shown that false assumptions in naturally occurring questions …
Question AnsweringPregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answering
Questions posed by information-seeking users often contain implicit false or potentially harmful assumptions. In a high-risk domain such as maternal and infant health, a question-answering system must recognize these pra…
Question AnsweringQuantum-Audit: Evaluating the Reasoning Limits of LLMs on Quantum Computing
Language models have become practical tools for quantum computing education and research, from summarizing technical papers to explaining theoretical concepts and answering questions about recent developments in the fiel…
Code GenerationWhat Patients Really Ask: Exploring the Effect of False Assumptions in Patient Information Seeking
Patients are increasingly using large language models (LLMs) to seek answers to their healthcare-related questions. However, benchmarking efforts in LLMs for question answering often focus on medical exam questions, whic…
Question Answering