StablePrompt: Automatic Prompt Tuning using Reinforcement Learning for Large Language Models
Finding appropriate prompts for the specific task has become an important issue as the usage of Large Language Models (LLM) has expanded. Reinforcement Learning (RL) is widely used for prompt tuning, but its inherent instability and environmental dependency make it difficult to use in practice. In this paper, we propose StablePrompt, which strikes a balance between training stability and search space, mitigating the instability of RL and producing high-performance prompts. We formulate prompt tuning as an online RL problem between the agent and target LLM and introduce Adaptive Proximal Policy Optimization (APPO). APPO introduces an LLM anchor model to adaptively adjust the rate of policy updates. This allows for flexible prompt search while preserving the linguistic ability of the pre-trained LLM. StablePrompt outperforms previous methods on various tasks including text classification, question answering, and text generation. Our code can be found in github.
Code (1)
Tasks
Question AnsweringReinforcement Learning (RL)text-classificationText ClassificationText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning to Rewrite Prompts for Personalized Text Generation
Facilitated by large language models (LLMs), personalized text generation has become a rapidly growing research direction. Most existing studies focus on designing specialized models for a particular domain, or they requ…
Language ModellingLarge Language Modelreinforcement-learningReinforcement Learning+2Optimizing Prompts for Text-to-Image Generation
Well-designed prompts can guide text-to-image models to generate amazing images. However, the performant prompts are often model-specific and misaligned with user input. Instead of laborious human engineering, we propose…
Language ModelingLanguage ModellingPrompt Engineeringreinforcement-learning+3RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
Prompting has shown impressive success in enabling large pretrained language models (LMs) to perform diverse NLP tasks, especially when only few downstream data are available. Automatically finding the optimal prompt for…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Style Transfer+2Automatic Label Sequence Generation for Prompting Sequence-to-sequence Models
Prompting, which casts downstream applications as language modeling tasks, has shown to be sample efficient compared to standard fine-tuning with pre-trained models. However, one pitfall of prompting is the need of manua…
Few-Shot LearningLanguage ModelingLanguage ModellingRe-RankingAutomatic Summarization of Doctor-Patient Encounter Dialogues Using Large Language Model through Prompt Tuning
Automatic text summarization (ATS) is an emerging technology to assist clinicians in providing continuous and coordinated care. This study presents an approach to summarize doctor-patient dialogues using generative large…
Language ModelingLanguage ModellingLarge Language ModelText Summarization