Continuous Prompt Generation from Linear Combination of Discrete Prompt Embeddings
The wayward quality of continuous prompts stresses the importance of their interpretability as unexpected and unpredictable behaviors appear following training, especially in the context of large language models automating people-sensitive tasks such as resume screening. In this paper we present a novel method of constructing continuous prompts via discrete prompt embeddings and evaluate improvements to continuous prompt interpretability and inference accuracy. For a set of manually designed discrete prompts $\mathcal{D}$, which we tokenize and embed each into tensor form, we train a model to predict the weights such that the linear combinations of those prompts correspond to higher performance on natural language understanding tasks.
Code (1)
Tasks
Natural Language UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
STEPS: Sequential Probability Tensor Estimation for Text-to-Image Hard Prompt Search
Recent text-to-image (T2I) diffusion models have demonstrated remarkable capabilities in visual synthesis, yet their performance heavily relies on the quality of input prompts. However, optimizing discrete prompts re…
Computational EfficiencyQuantizationDialogue for Prompting: a Policy-Gradient-Based Discrete Prompt Generation for Few-shot Learning
Prompt-based pre-trained language models (PLMs) paradigm have succeeded substantially in few-shot natural language processing (NLP) tasks. However, prior discrete prompt optimization methods require expert knowledge to d…
Few-Shot LearningReinforcement Learning (RL)Context-Tuning: Learning Contextualized Prompts for Natural Language Generation
Recently, pretrained language models (PLMs) have had exceptional success in language generation. To leverage the rich knowledge encoded by PLMs, a simple yet powerful paradigm is to use prompts in the form of either disc…
Text GenerationThe importance of being discrete: on the inaccuracy of continuous approximations in auction theory
While auction theory views bids and valuations as continuous variables, real-world auctions are necessarily discrete. In this paper, we use a combination of analytical and computational methods to investigate whether inc…
Towards Diverse, Relevant and Coherent Open-Domain Dialogue Generation via Hybrid Latent Variables
Conditional variational models, using either continuous or discrete latent variables, are powerful for open-domain dialogue response generation. However, previous works show that continuous latent variables tend to reduc…
Dialogue GenerationResponse Generation