Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning
Self-consistency (SC) is a popular technique for improving the reasoning accuracy of large language models by aggregating multiple sampled outputs, but it comes at a high computational cost due to extensive sampling. We introduce a hybrid ensembling approach that leverages the complementary strengths of two distinct modes of reasoning: Chain-of-Thought (CoT) and Program-of-Thought (PoT). We describe a general framework for combining these two forms of reasoning in self-consistency, as well as particular strategies for both full sampling and early-stopping. We show that CoT-PoT ensembling not only improves overall accuracy, but also drastically reduces the number of samples required for SC by a factor of 9.3x. In particular, the majority of tasks (78.6%) can be addressed with only two samples, which has not been possible with any prior SC methods.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
PEDAL: Enhancing Greedy Decoding with Large Language Models using Diverse Exemplars
Self-ensembling techniques with diverse reasoning paths such as Self-Consistency have demonstrated remarkable performance gains in text generation with Large Language Models (LLMs). However, such techniques depend on the…
ARCDiversityText GenerationSemi-supervised Medical Image Classification with Relation-driven Self-ensembling Model
Training deep neural networks usually requires a large amount of labeled data to obtain good performance. However, in medical image analysis, obtaining high-quality labels for the data is laborious and expensive, as accu…
ClassificationGeneral Classificationimage-classificationImage Classification+5The Impact of LLM Self-Consistency and Reasoning Effort on Automated Scoring Accuracy and Cost
Strategic model selection and reasoning settings are more effective than ensembling for optimizing automated scoring with large language models (LLMs). We examined self-consistency (intra-model majority voting) and reaso…
High School MathematicsACAR: Adaptive Complexity Routing for Multi-Model Ensembles with Auditable Decision Traces
We present ACAR (Adaptive Complexity and Attribution Routing), a measurement framework for studying multi-model orchestration under auditable conditions. ACAR uses self-consistency variance (sigma) computed from N=3 prob…
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the …
ARCArithmetic ReasoningGSM8KLanguage Modelling+2