Optimizing Decomposition for Optimal Claim Verification
Current research on the \textit{Decompose-Then-Verify} paradigm for evaluating the factuality of long-form text typically treats decomposition and verification in isolation, overlooking their interactions and potential misalignment. We find that existing decomposition policies, typically hand-crafted demonstrations, do not align well with downstream verifiers in terms of atomicity -- a novel metric quantifying information density -- leading to suboptimal verification results. We formulate finding the optimal decomposition policy for optimal verification as a bilevel optimization problem. To approximate a solution for this strongly NP-hard problem, we propose dynamic decomposition, a reinforcement learning framework that leverages verifier feedback to learn a policy for dynamically decomposing claims to verifier-preferred atomicity. Experimental results show that dynamic decomposition outperforms existing decomposition policies, improving verification confidence by 0.07 and accuracy by 0.12 (on a 0-1 scale) on average across varying verifiers, datasets, and atomcities of input claims.
Code (1)
Tasks
Bilevel OptimizationClaim VerificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Distill and Align Decomposition for Enhanced Claim Verification
Complex claim verification requires decomposing sentences into verifiable subclaims, yet existing methods struggle to align decomposition quality with verification performance. We propose a reinforcement learning (RL) ap…
Reinforcement LearningFrom Verdict to Process: Agentic Reinforcement Learning for Multi-Stage Fact Verification
Recent approaches combining Large Language Models (LLMs) with retrieval-augmented reasoning have shown promise for automated fact verification. To process complex claims, these verification pipelines typically execute mu…
Reinforcement LearningFact VerificationAnswer GenerationDnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generation
The decompose-then-verify strategy for verification of Large Language Model (LLM) generations decomposes claims that are then independently verified. Decontextualization augments text (claims) to ensure it can be verifie…
FormLanguage ModelingLanguage ModellingLarge Language Model+1Reasoning-CV: Fine-tuning Powerful Reasoning LLMs for Knowledge-Assisted Claim Verification
Claim verification is essential in combating misinformation, and large language models (LLMs) have recently emerged in this area as powerful tools for assessing the veracity of claims using external knowledge. Existing L…
Claim VerificationMisinformationA Graph-based Verification Framework for Fact-Checking
Fact-checking plays a crucial role in combating misinformation. Existing methods using large language models (LLMs) for claim decomposition face two key limitations: (1) insufficient decomposition, introducing unnecessar…
Fact Checkinggraph constructionMisinformationTriplet