Prefix-Tuning Based Unsupervised Text Style Transfer
Unsupervised text style transfer aims at training a generative model that can alter the style of the input sentence while preserving its content without using any parallel data. In this paper, we employ powerful pre-trained large language models and present a new prefix-tuning-based method for unsupervised text style transfer. We construct three different kinds of prefixes, i.e., \textit{shared prefix, style prefix}, and \textit{content prefix}, to encode task-specific information, target style, and the content information of the input sentence, respectively. Compared to embeddings used by previous works, the proposed prefixes can provide richer information for the model. Furthermore, we adopt a recursive way of using language models in the process of style transfer. This strategy provides a more effective way for the interactions between the input sentence and GPT-2, helps the model construct more informative prefixes, and thus, helps improve the performance. Evaluations on the well-known datasets show that our method outperforms the state-of-the-art baselines. Results, analysis of ablation studies, and subjective evaluations from humans are also provided for a deeper understanding of the proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceStyle TransferText Style TransferUnsupervised Text Style TransferMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Inducer-tuning: Connecting Prefix-tuning and Adapter-tuning
Prefix-tuning, or more generally continuous prompt tuning, has become an essential paradigm of parameter-efficient transfer learning. Using a large pre-trained language model (PLM), prefix-tuning can obtain strong perfor…
Language ModelingLanguage ModellingNatural Language UnderstandingTransfer LearningDAML-ST5: Low Resource Style Transfer via Domain Adaptive Meta Learning
Text style transfer (TST) without parallel data has achieved some practical success. However, most of the existing unsupervised text style transfer methods suffer from (i) requiring massive amounts of nonparallel data to…
General KnowledgeLanguage ModelingLanguage ModellingMeta-Learning+3Low Resource Style Transfer via Domain Adaptive Meta Learning
Text style transfer (TST) without parallel data has achieved some practical success. However, most of the existing unsupervised text style transfer methods suffer from (i) requiring massive amounts of nonparallel data t…
General KnowledgeLanguage ModelingLanguage ModellingMeta-Learning+3Low Resource Style Transfer via Domain Adaptive Meta Learning
Text style transfer (TST) without parallel data has achieved some practical success. However, most of the existing unsupervised text style transfer methods suffer from (i) requiring massive amounts of non-parallel data t…
General KnowledgeLanguage ModelingLanguage ModellingMeta-Learning+3The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models
Improving the reasoning capabilities of large language models (LLMs) typically requires supervised fine-tuning with labeled data or computationally expensive sampling. We introduce Unsupervised Prefix Fine-Tuning (UPFT),…