Probabilistic Soundness Guarantees in LLM Reasoning Chains

2025-07-17 · Weiqiu You, Anton Xue, Shreya Havaldar, Delip Rao, Helen Jin, Chris Callison-Burch, Eric Wong arxiv

In reasoning chains generated by large language models (LLMs), initial errors often propagate and undermine the reliability of the final conclusion. Current LLM-based error detection methods often fail to detect propagated errors because earlier errors can corrupt judgments of downstream reasoning. To better detect such errors, we introduce Autoregressive Reasoning Entailment Stability (ARES), a probabilistic framework that evaluates each reasoning step based solely on previously-verified premises. This inductive method yields a nuanced score for each step and provides certified statistical guarantees of its soundness, rather than a brittle binary label. ARES achieves state-of-the-art performance across four benchmarks (72.1% Macro-F1, +8.2 points) and demonstrates superior robustness on very long synthetic reasoning chains, where it excels at detecting propagated errors (90.3% F1, +27.6 points).

📄 PDF Abstract BibTeX arXiv:2507.12948

Code (0)

등록된 구현이 없습니다.

Tensor Probabilistic Model Checking of Finite-Horizon Markov Chains (Extended Version)

2026-08-01 · Jianlin Li, Nick Guo, Peter Ye, Yizhou Zhang arxiv

We reexamine the problem of verifying Markov chains with respect to step-bounded reachability probabilities. Prevailing approaches rely on encoding the state-transition matrix using either explicit or symbolic representa…

Soundness-Aware Level: A Microscopic Signature that Predicts LLM Reasoning Potential

2025-10-17 · Xuansheng Wu, Xiaoman Pan, Wenlin Yao, Jianshu Chen arxiv

Reinforcement learning with verifiable rewards (RLVR) can elicit strong reasoning in large language models (LLMs), while their performance after RLVR varies dramatically across different base models. This raises a fundam…

Reinforcement Learning

Investigating the Shortcomings of LLMs in Step-by-Step Legal Reasoning

2025-02-08 · Venkatesh Mishra, Bimsara Pathiraja, Mihir Parmar, Sat Chidananda 외

Reasoning abilities of LLMs have been a key focus in recent years. One challenging reasoning domain with interesting nuances is legal reasoning, which requires careful application of rules, and precedents while balancing…

Legal Reasoning Multiple-choice Multiple Choice Question Answering (MCQA)Question Answering

Multi-Source Evidence Fusion for Audio Question Answering

2026-03-18 · Aivo Olev, Tanel Alumäe arxiv

Large audio language models (LALMs) can answer questions about speech, music, and environmental sounds, yet their internal reasoning is largely opaque and difficult to validate. We describe TalTech's solution to the Agen…

Question Answering

Approximating Probabilistic Inference in Statistical EL with Knowledge Graph Embeddings

2024-07-16 · Yuqicheng Zhu, Nico Potyka, Bo Xiong, Trung-Kien Tran 외

Statistical information is ubiquitous but drawing valid conclusions from it is prohibitively hard. We explain how knowledge graph embeddings can be used to approximate probabilistic inference efficiently using the exampl…

Knowledge Graph Embeddings valid