Asking the Right Questions: Improving Reasoning with Generated Stepping Stones
Recent years have witnessed tremendous progress in enabling LLMs to solve complex reasoning tasks such as math and coding. As we start to apply LLMs to harder tasks that they may not be able to solve in one shot, it is worth paying attention to their ability to construct intermediate stepping stones that prepare them to better solve the tasks. Examples of stepping stones include simplifications, alternative framings, or subproblems. We study properties and benefits of stepping stones in the context of modern reasoning LLMs via ARQ (Asking the Right Questions), a simple framework that introduces a question generator to the default reasoning pipeline. We first show that good stepping stone questions exist and are transferrable, meaning that good questions can be generated, and they substantially help LLMs of various capabilities in solving the target tasks. We next frame stepping stone generation as a post-training task and show that we can fine-tune LLMs to generate more useful stepping stones by SFT and RL on synthetic data.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Asking Complex Questions with Multi-hop Answer-focused Reasoning
Asking questions from natural language text has attracted increasing attention recently, and several schemes have been proposed with promising results by asking the right question words and copy relevant words from the i…
Question GenerationQuestion-GenerationAre You Asking the Right Questions? Teaching Machines to Ask Clarification Questions
A Feasibility Study of Answer-Unaware Question Generation for Education
We conduct a feasibility study into the applicability of answer-unaware question generation models to textbook passages. We show that a significant portion of errors in such systems arise from asking irrelevant or un-int…
Question GenerationQuestion-GenerationA Feasibility Study of Answer-Agnostic Question Generation for Education
We conduct a feasibility study into the applicability of answer-agnostic question generation models to textbook passages. We show that a significant portion of errors in such systems arise from asking irrelevant or unint…
Question GenerationQuestion-GenerationSocratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions
Recent research in vision-language models (VLMs) has centered around the possibility of equipping them with implicit long-form chain-of-thought reasoning -- akin to the success observed in language models -- via distilla…
Visual Reasoning