Unsupervised Text Generation by Learning from Search
In this work, we present TGLS, a novel framework to unsupervised Text Generation by Learning from Search. We start by applying a strong search algorithm (in particular, simulated annealing) towards a heuristically defined objective that (roughly) estimates the quality of sentences. Then, a conditional generative model learns from the search results, and meanwhile smooth out the noise of search. The alternation between search and learning can be repeated for performance bootstrapping. We demonstrate the effectiveness of TGLS on two real-world natural language generation tasks, paraphrase generation and text formalization. Our model significantly outperforms unsupervised baseline methods in both tasks. Especially, it achieves comparable performance with the state-of-the-art supervised methods in paraphrase generation.
Code (0)
등록된 구현이 없습니다.
Tasks
Paraphrase GenerationText GenerationSimilar Papers 제목 키워드 기반
GenWiki: A Dataset of 1.3 Million Content-Sharing Text and Graphs for Unsupervised Graph-to-Text Generation
Data collection for the knowledge graph-to-text generation is expensive. As a result, research on unsupervised models has emerged as an active field recently. However, most unsupervised models have to use non-parallel ve…
Knowledge GraphsText GenerationUnsupervised KG-to-Text GenerationEffective Unsupervised Constrained Text Generation based on Perturbed Masking
Unsupervised constrained text generation aims to generate text under a given set of constraints without any supervised data. Current state-of-the-art methods stochastically sample edit positions and actions, which may ca…
PositionSentenceText GenerationEffective Unsupervised Constrained Text Generation based on Perturbed Masking
Unsupervised constrained text generation aims to generate text under a given set of constraints without any supervised data. Current state-of-the-art methods stochastically sample edit positions which may cause unnecessa…
PositionSentenceText GenerationSearch and Learning for Unsupervised Text Generation
With the advances of deep learning techniques, text generation is attracting increasing interest in the artificial intelligence (AI) community, because of its wide applications and because it is an essential component of…
SentenceText GenerationLearning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization
Text summarization aims to generate a short summary for an input text. In this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS) approach, which does not require parallel data for training. Our NAUS…
Headline GenerationSentenceSentence SummarizationText Summarization+1