Boosting Process-Correct CoT Reasoning by Modeling Solvability of Multiple-Choice QA
Reasoning quality in large language models depends not only on producing correct answers but also on generating valid intermediate steps. We study this through multiple-choice question answering (MCQA), which provides a controlled setting with fixed answer options. Our analysis shows that when questions are effectively unsolvable for a model, spurious chains of thought (CoTs) are more likely to appear, leading to false positives. By estimating the solvability of each question, we uncover an intermediate regime where learning is most effective. Building on this insight, we adapt outcome-supervised reward models and reinforcement learning with group-relative advantage to incorporate solvability into their objectives. Across experiments on math and multimodal datasets, these modifications consistently yield higher rates of process-correct reasoning and, in reinforcement learning, improved answer accuracy as well. Our results highlight solvability as a key factor for reducing hallucinations and increasing reliability in CoT reasoning.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningQuestion AnsweringSimilar Papers 제목 키워드 기반
MiST: Understanding the Role of Mid-Stage Scientific Training in Developing Chemical Reasoning Models
Large Language Models can develop reasoning capabilities through online fine-tuning with rule-based rewards. However, recent studies reveal a critical constraint: reinforcement learning succeeds only when the base model …
Reinforcement LearningMore Capable, Less Faithful: A Multilingual Analysis of Mathematical (Un)Solvability Detection in LLMs
Solvability detection is one of the most challenging aspects of mathematical reasoning for Large Language Models (LLMs). While prior work has studied this capability extensively, these analyses have been limited to Engli…
Mathematical ReasoningCode-driven Number Sequence Calculation: Enhancing the inductive Reasoning Abilities of Large Language Models
Large language models (LLMs) make remarkable progress in reasoning tasks. Among different reasoning modes, inductive reasoning, due to its better alignment with human learning, attracts increasing interest. However, rese…
Reinforcement LearningLearning the Boundary of Solvability: Aligning LLMs to Detect Unsolvable Problems
Ensuring large language model (LLM) reliability requires distinguishing objective unsolvability (inherent contradictions) from subjective capability limitations (tasks exceeding model competence). Current LLMs often conf…
Reinforcement LearningKnowledge Knows, Verbalization Tells: Disentangling Latent Directions for Mathematical Solvability in LLMs
Although LLMs have made significant progress in mathematical reasoning, determining whether a mathematical problem is solvable remains a fundamental yet challenging capability. While recent studies have probed internal r…
Mathematical Reasoning