ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents

2026-06-16 · Ander Alvarez, Santhiya Rajan, Samuel Mugel, Román Orús arxiv

Tool-using LLM agents increasingly use the Model Context Protocol (MCP) to answer from heterogeneous evidence sources, including search, APIs, databases, clinical records, and formulary tools. Standard factuality metrics usually test whether an answer is supported by pooled evidence, missing a provenance-sensitive failure mode: a claim may be supported somewhere while being attributed to the wrong source. We call this cross-source conflation. We introduce ProvenanceGuard, a source-aware verifier for MCP-grounded answers. It consumes captured MCP traces with stable tool IDs, source IDs, and raw outputs; decomposes answers into atomic claims; routes claims to source-specific evidence; checks support with NLI and a token-alignment proxy; compares stated attribution with the routed source; and returns per-claim verdicts plus an answer-level allow/block decision. Blocked answers can be repaired with retrieval-augmented answer revision and re-verified. We evaluate on 281 medical-domain MCP-agent traces. A 266-trace adjudicated subset yields 2,325 LLM-assisted claim labels split by trace; 361 held-out labels are human-verified. On the 40-trace held-out split, ProvenanceGuard achieves block F1 0.802 and source accuracy 0.858 over 260 source-eligible claims, outperforming source-blind baselines that do not emit claim-to-source IDs. On a harder multi-source benchmark it reaches block F1 0.846, while source-plus-relation accuracy drops to 0.229, showing that exact source ownership remains difficult with semantically close sources. Repair-and-reverify resolves all blocked answers in the full trace set, often via conservative fallback. In 50 controlled clinical conflation probes, ProvenanceGuard detects all injected attribution swaps with no retained wrong attribution. These results show that source attribution is an independent axis for factuality verification in MCP-based agents.

📄 PDF Abstract BibTeX arXiv:2606.18037

Code (0)

등록된 구현이 없습니다.

ElementCheck: Complexity-Aware Long-Form Text Factuality Evaluation via Sentence Elements

2026-06-17 · Xinming Wang, Haoran Du, Yi Chen, Jian Xu 외 arxiv

Existing long-form factuality evaluation relies on the decompose-retrieve-verify pipeline. However, the pipeline suffers from noise from claim decomposition and fixed verification granularity, resulting in unreliable res…

Not All Claims Are Equally Risky: FACTOR for Adaptive Verification in Factual Long-Form Generation

2026-06-21 · Areeba Hassan, Arooj Kausar, Syeda Kisaa Fatima, Gibrail Islam 외 arxiv

Large Language Models (LLMs) generate fluent long-form text, however, often add unsupported factual claims. Existing verification techniques improve factuality by grounding generation in external evidence. However, the s…

AEScorer: An Agentic Evidence-Grounded Framework for Graded Factuality Verification

2026-01-07 · Hui Huang, Muyun Yang, Yuki Arase arxiv

Despite the significant advancements of Large Language Models (LLMs), their factuality remains a critical challenge, creating a growing need for more nuanced factuality verification. Existing factuality verification meth…

Truth or Mirage? Towards End-to-End Factuality Evaluation with LLM-Oasis

2024-11-29 · Alessandro Scirè, Andrei Stefan Bejgu, Simone Tedeschi, Karim Ghonim 외

After the introduction of Large Language Models (LLMs), there have been substantial improvements in the performance of Natural Language Generation (NLG) tasks, including Text Summarization and Machine Translation. Howeve…

Benchmarking Claim Verification Fact Checking Fact Verification+1

Safeguarding LLM Agents from Misalignment through Provenance Analysis

2026-05-01 · Yining She, Yiliang Liang, Eunsuk Kang arxiv

As LLM agents gain increasing access to powerful tools, ensuring that their actions are aligned with the user's intent becomes critical. When an agent's proposed tool invocation deviates from the user's intent -- a pheno…