Discrete Structural Planning for Generating Diverse Translations
Planning is important for humans when producing complex languages, which is a missing part in current language generation models. In this work, we add a planning phase in neural machine translation to control the global sentence structure ahead of translation. Our approach learns discrete structural representations to encode syntactic information of target sentences. During translation, we can either let beam search to choose the structural codes automatically or specify the codes manually. The word generation is then conditioned on the selected discrete codes. Experiments show that the translation performance remains intact by learning the codes to capture pure structural variations. Through structural planning, we are able to control the global sentence structure by manipulating the codes. By evaluating with a proposed structural diversity metric, we found that the sentences sampled using different codes have much higher diversity scores. In qualitative analysis, we demonstrate that the sampled paraphrase translations have drastically different structures.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityMachine TranslationSentenceText GenerationTranslationSimilar Papers 제목 키워드 기반
Discrete Structural Planning for Neural Machine Translation
Structural planning is important for producing long sentences, which is a missing part in current language generation models. In this work, we add a planning phase in neural machine translation to control the coarse stru…
Machine TranslationSentenceText GenerationTranslationDiscrete Structural Planning for Neural Machine Translation
Structural planning is important for producing long sentences, which is a missing part in current language generation models. In this work, we add a planning phase in neural machine translation to control the coarse stru…
Machine TranslationSentenceText GenerationTranslationBridging the Gap between Structural and Semantic Similarity in Diverse Planning
Diverse planning is the problem of finding multiple plans for a given problem specification, which is at the core of many real-world applications. For example, diverse planning is a critical piece for the efficiency of p…
Semantic SimilaritySemantic Textual SimilarityDiscrete Autoencoders for Sequence Models
Recurrent models for sequences have been recently successful at many tasks, especially for language modeling and machine translation. Nevertheless, it remains challenging to extract good representations from these models…
Language ModelingLanguage ModellingMachine TranslationTranslationGenerating Diverse Translation by Manipulating Multi-Head Attention
Transformer model has been widely used on machine translation tasks and obtained state-of-the-art results. In this paper, we report an interesting phenomenon in its encoder-decoder multi-head attention: different attenti…
Data AugmentationDecoderDiversityMachine Translation+3