LAMBADA: Backward Chaining for Automated Reasoning in Natural Language
Remarkable progress has been made on automated reasoning with natural text, by using Language Models (LMs) and methods such as Chain-of-Thought and Selection-Inference. These techniques search for proofs in the forward direction from axioms to the conclusion, which suffers from a combinatorial explosion of the search space, and thus high failure rates for problems requiring longer chains of reasoning. The classical automated reasoning literature has shown that reasoning in the backward direction (i.e. from the intended conclusion to supporting axioms) is significantly more efficient at proof-finding. Importing this intuition into the LM setting, we develop a Backward Chaining algorithm, called LAMBADA, that decomposes reasoning into four sub-modules. These sub-modules are simply implemented by few-shot prompted LM inference. We show that LAMBADA achieves sizable accuracy boosts over state-of-the-art forward reasoning methods on challenging logical reasoning datasets, particularly when deep and accurate proof chains are required.
Code (0)
등록된 구현이 없습니다.
Tasks
LAMBADALogical ReasoningSimilar Papers 제목 키워드 기반
SymBa: Symbolic Backward Chaining for Structured Natural Language Reasoning
To improve the performance and explainability of LLM-based natural language reasoning, structured reasoning can be applied to generate explicitly structured proofs. Among different methods for structured reasoning, we sp…
Arithmetic ReasoningGSM8KLAMBADAScheherazade: Evaluating Chain-of-Thought Math Reasoning in LLMs with Chain-of-Problems
Benchmarks are critical for measuring progress of math reasoning abilities of Large Language Models (LLMs). However, existing widely-used benchmarks such as GSM8K have been rendered less useful as multiple cutting-edge L…
GSM8KMathMathematical ReasoningBi-Chainer: Automated Large Language Models Reasoning with Bidirectional Chaining
Large Language Models (LLMs) have shown human-like reasoning abilities but still face challenges in solving complex logical problems. Existing unidirectional chaining methods, such as forward chaining and backward chaini…
Logical ReasoningLarge-Scale Reasoning with OWL
With the growth of the Semantic Web in size and importance, more and more knowledge is stored in machine-readable formats such as the Web Ontology Language OWL. This paper outlines common approaches for efficient reasoni…
Evaluating Step-by-Step Reasoning through Symbolic Verification
Pre-trained language models (LMs) have shown remarkable reasoning performance using explanations or chain-of-thoughts (CoT)) for in-context learning. On the other hand, these reasoning tasks are usually presumed to be mo…
In-Context Learning