Step-by-Step Fact Verification System for Medical Claims with Explainable Reasoning
Fact verification (FV) aims to assess the veracity of a claim based on relevant evidence. The traditional approach for automated FV includes a three-part pipeline relying on short evidence snippets and encoder-only inference models. More recent approaches leverage the multi-turn nature of LLMs to address FV as a step-by-step problem where questions inquiring additional context are generated and answered until there is enough information to make a decision. This iterative method makes the verification process rational and explainable. While these methods have been tested for encyclopedic claims, exploration on domain-specific and realistic claims is missing. In this work, we apply an iterative FV system on three medical fact-checking datasets and evaluate it with multiple settings, including different LLMs, external web search, and structured reasoning using logic predicates. We demonstrate improvements in the final performance over traditional approaches and the high potential of step-by-step FV systems for domain-specific claims.
Code (1)
Tasks
Fact CheckingFact VerificationSimilar Papers 제목 키워드 기반
Uncertainty Estimation of Large Language Models in Medical Question Answering
Large Language Models (LLMs) show promise for natural language generation in healthcare, but risk hallucinating factually incorrect information. Deploying LLMs for medical question answering necessitates reliable uncerta…
Medical Question AnsweringQuestion AnsweringText GenerationGrading the Graders: Verification Autonomy Levels (L0-L5) for LLM Reasoning
Large language models (LLMs) are increasingly paired with verifiers (step checkers, self-consistency filters, tool-based fact checkers, formal proof assistants) that claim to detect the model's errors. Yet the verificati…
Medical DiagnosisCode GenerationMedScore: Factuality Evaluation of Free-Form Medical Answers
While Large Language Models (LLMs) can generate fluent and convincing responses, they are not necessarily correct. This is especially apparent in the popular decompose-then-verify factuality evaluation pipeline, where LL…
FormHallucinationSentencevalidChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification
Recent advances in reasoning-enhanced large language models (LLMs) and multimodal LLMs (MLLMs) have significantly improved performance in complex tasks, yet medical AI models often overlook the structured reasoning proce…
DiagnosticQuestion AnsweringVisual Question AnsweringScientific Claim Verification with VERT5ERINI
This work describes the adaptation of a pretrained sequence-to-sequence model to the task of scientific claim verification in the biomedical domain. We propose VERT5ERINI that exploits T5 for abstract retrieval, sentence…
Claim VerificationRetrievalSentence