Uncovering Latent Chain of Thought Vectors in Language Models
As language models grow more influential and trusted in our society, our ability to reliably steer them toward favorable behaviors becomes increasingly paramount. For this, we investigate the technique of steering vectors: biasing the forward pass of language models using a "steering vector" derived from a specific task. We apply them to steer language models toward performing Chain of Thought (CoT) Reasoning without the need to prompt through natural language. We demonstrate this approach on Llama3 8b and Mistral 7b v0.2, and obtain competitive results compared to CoT-prompted performances on a series of reasoning benchmarks (GSM8k, MMLU, AGI Eval, ARC AI2) and qualitative examples. We find this approach yields consistent steering towards CoT responses and takes less compute than traditional methods of fine-tuning models towards CoT.
Code (0)
등록된 구현이 없습니다.
Tasks
ARCGSM8KMMLUSimilar Papers 제목 키워드 기반
Do Latent Tokens Think? A Causal and Adversarial Analysis of Chain-of-Continuous-Thought
Latent tokens are gaining attention for enhancing reasoning in large language models (LLMs), yet their internal mechanisms remain unclear. This paper examines the problem from a reliability perspective, uncovering fundam…
Steering Risk Preferences in Large Language Models by Aligning Behavioral and Neural Representations
Changing the behavior of large language models (LLMs) can be as straightforward as editing the Transformer's residual streams using appropriately constructed "steering vectors." These modifications to internal neural act…
Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization
Recent advancements in Large Language Models (LLMs) have shifted from explicit Chain-of-Thought (CoT) reasoning to more efficient latent reasoning, where intermediate thoughts are represented as vectors rather than text.…
Text GenerationUnlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention
Latent reasoning enables Large Language Models (LLMs) to perform multi-step inference within continuous hidden states, offering efficiency gains over explicit Chain-of-Thought (CoT). However, the opacity of these continu…
Inference-Time Rethinking with Latent Thought Vectors for Math Reasoning
Standard chain-of-thought reasoning generates a solution in a single forward pass, committing irrevocably to each token and lacking a mechanism to recover from early errors. We introduce Inference-Time Rethinking, a gene…
Mathematical Reasoning