Boosting Deductive Reasoning with Step Signals In RLHF
Logical reasoning is a crucial task for Large Language Models (LLMs), enabling them to tackle complex problems. Among reasoning tasks, multi-step reasoning poses a particular challenge. Grounded in the theory of formal logic, we have developed an automated method, Multi-step Deduction (MuseD), for deductive reasoning data. MuseD has allowed us to create training and testing datasets for multi-step reasoning. Our generation method enables control over the complexity of the generated instructions, facilitating training and evaluation of models across different difficulty levels. Through RLHF training, our training data has demonstrated significant improvements in logical capabilities for both in-domain of out-of-domain reasoning tasks. Additionally, we have conducted tests to assess the multi-step reasoning abilities of various models.
Code (0)
등록된 구현이 없습니다.
Tasks
Formal LogicLogical ReasoningSimilar Papers 제목 키워드 기반
Deductive Verification of Chain-of-Thought Reasoning
Large Language Models (LLMs) significantly benefit from Chain-of-Thought (CoT) prompting in performing various reasoning tasks. While CoT allows models to produce more comprehensive reasoning processes, its emphasis on i…
Logical ReasoningHow Clued up are LLMs? Evaluating Multi-Step Deductive Reasoning in a Text-Based Game Environment
Deducing whodunit proves challenging for LLM agents. In this paper, we implement a text-based multi-agent version of the classic board game Clue as a rule-based testbed for evaluating multi-step deductive reasoning, with…
Boosting Theory-of-Mind Performance in Large Language Models via Prompting
Large language models (LLMs) excel in many tasks in 2023, but they still face challenges in complex reasoning. Theory-of-mind (ToM) tasks, which require understanding agents' beliefs, goals, and mental states, are essent…
Common Sense ReasoningIn-Context LearningLearning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction
Solving math word problems requires deductive reasoning over the quantities in the text. Various recent research efforts mostly relied on sequence-to-sequence or sequence-to-tree models to generate mathematical expressio…
MathMath Word Problem SolvingRelationRelational Reasoning+1A criterion for Artificial General Intelligence: hypothetic-deductive reasoning, tested on ChatGPT
We argue that a key reasoning skill that any advanced AI, say GPT-4, should master in order to qualify as 'thinking machine', or AGI, is hypothetic-deductive reasoning. Problem-solving or question-answering can quite gen…
ChatbotQuestion Answering