Search and Learning for Unsupervised Text Generation
With the advances of deep learning techniques, text generation is attracting increasing interest in the artificial intelligence (AI) community, because of its wide applications and because it is an essential component of AI. Traditional text generation systems are trained in a supervised way, requiring massive labeled parallel corpora. In this paper, I will introduce our recent work on search and learning approaches to unsupervised text generation, where a heuristic objective function estimates the quality of a candidate sentence, and discrete search algorithms generate a sentence by maximizing the search objective. A machine learning model further learns from the search results to smooth out noise and improve efficiency. Our approach is important to the industry for building minimal viable products for a new task; it also has high social impacts for saving human annotation labor and for processing low-resource languages.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceText GenerationSimilar Papers 제목 키워드 기반
GenWiki: A Dataset of 1.3 Million Content-Sharing Text and Graphs for Unsupervised Graph-to-Text Generation
Data collection for the knowledge graph-to-text generation is expensive. As a result, research on unsupervised models has emerged as an active field recently. However, most unsupervised models have to use non-parallel ve…
Knowledge GraphsText GenerationUnsupervised KG-to-Text GenerationUnsupervised Text Generation by Learning from Search
In this work, we present TGLS, a novel framework to unsupervised Text Generation by Learning from Search. We start by applying a strong search algorithm (in particular, simulated annealing) towards a heuristically define…
Paraphrase GenerationText GenerationEffective Unsupervised Constrained Text Generation based on Perturbed Masking
Unsupervised constrained text generation aims to generate text under a given set of constraints without any supervised data. Current state-of-the-art methods stochastically sample edit positions and actions, which may ca…
PositionSentenceText GenerationEffective Unsupervised Constrained Text Generation based on Perturbed Masking
Unsupervised constrained text generation aims to generate text under a given set of constraints without any supervised data. Current state-of-the-art methods stochastically sample edit positions which may cause unnecessa…
PositionSentenceText GenerationLearning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization
Text summarization aims to generate a short summary for an input text. In this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS) approach, which does not require parallel data for training. Our NAUS…
Headline GenerationSentenceSentence SummarizationText Summarization+1