홈 › Papers

Evergreen: Efficient Claim Verification for Semantic Aggregates

With recent semantic query processing engines, semantic aggregation has become a primitive operator, enabling the reduction of a relation into a natural language aggregate using an LLM. However, the resulting semantic aggregate may contain claims that are not grounded in the underlying relation. Verifying such claims is challenging: they often involve quantifiers, groupings, and comparisons over relations that far exceed LLM context windows and require a costly combination of semantic and symbolic processing. We present Evergreen, a system that recasts claim verification as a semantic query processing task with tailored optimizations and provenance capture. Evergreen compiles each claim into a declarative semantic verification query that can execute on the same query engine used to produce the aggregate. To reduce cost, Evergreen avoids unnecessary LLM calls through verification-aware optimizations, including early stopping, relevance sorting, and estimation with confidence sequences, as well as general-purpose optimizations for semantic queries, such as operator fusion, similarity filtering, and prompt caching. Each verdict is accompanied by citations that identify a minimal set of tuples justifying the result, with semantics based on semiring provenance for first-order logic. On a benchmark of production-inspired workloads over restaurant review and customer support datasets, Evergreen's optimized configurations occupy the entire cost-quality Pareto frontier. With a strong LLM, Evergreen preserves verification quality at an F1 of 0.94 while reducing cost by 3.1x relative to unoptimized verification; with a substantially weaker LLM, it surpasses the strongest external baseline's F1 (0.87 vs. 0.83) at 7.0x lower cost.

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반