WST: Weak-to-Strong Knowledge Transfer via Reinforcement Learning
Effective prompt engineering remains a challenging task for many applications. We introduce Weak-to-Strong Transfer (WST), an automatic prompt engineering framework where a small "Teacher" model generates instructions that enhance the performance of a much larger "Student" model. Unlike prior work, WST requires only a weak teacher, making it efficient and broadly applicable in settings where large models are closed-source or difficult to fine-tune. Using reinforcement learning, the Teacher Model's instructions are iteratively improved based on the Student Model's outcomes, yielding substantial gains across reasoning (MATH-500, GSM8K) and alignment (HH-RLHF) benchmarks - 98% on MATH-500 and 134% on HH-RLHF - and surpassing baselines such as GPT-4o-mini and Llama-70B. These results demonstrate that small models can reliably scaffold larger ones, unlocking latent capabilities while avoiding misleading prompts that stronger teachers may introduce, establishing WST as a scalable solution for efficient and safe LLM prompt refinement.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningPrompt EngineeringSimilar Papers 제목 키워드 기반
A transfer learning framework for weak-to-strong generalization
Modern large language model (LLM) alignment techniques rely on human feedback, but it is unclear whether these techniques fundamentally limit the capabilities of aligned LLMs. In particular, it is unknown if it is possib…
Language ModelingLanguage ModellingLarge Language ModelTransfer LearningDetector Discovery in the Wild: Joint Multiple Instance and Representation Learning
We develop methods for detector learning which exploit joint training over both weak and strong labels and which transfer learned perceptual representations from strongly-labeled auxiliary tasks. Previous methods for wea…
Multiple Instance LearningRepresentation LearningTransfer LearningTransferable Model-agnostic Vision-Language Model Adaptation for Efficient Weak-to-Strong Generalization
Vision-Language Models (VLMs) have been widely used in various visual recognition tasks due to their remarkable generalization capabilities. As these models grow in size and complexity, fine-tuning becomes costly, emphas…
Can a powerful neural network be a teacher for a weaker neural network?
The transfer learning technique is widely used to learning in one context and applying it to another, i.e. the capacity to apply acquired knowledge and skills to new situations. But is it possible to transfer the learnin…
Transfer LearningCombining inherent knowledge of vision-language models with unsupervised domain adaptation through strong-weak guidance
Unsupervised domain adaptation (UDA) tries to overcome the tedious work of labeling data by leveraging a labeled source dataset and transferring its knowledge to a similar but different target dataset. Meanwhile, current…
Domain AdaptationKnowledge DistillationLanguage ModellingSelf-Knowledge Distillation+1