A Tree-based Decoder for Neural Machine Translation
Recent advances in Neural Machine Translation (NMT) show that adding syntactic information to NMT systems can improve the quality of their translations. Most existing work utilizes some specific types of linguistically-inspired tree structures, like constituency and dependency parse trees. This is often done via a standard RNN decoder that operates on a linearized target tree structure. However, it is an open question of what specific linguistic formalism, if any, is the best structural representation for NMT. In this paper, we (1) propose an NMT model that can naturally generate the topology of an arbitrary tree structure on the target side, and (2) experiment with various target tree structures. Our experiments show the surprising result that our model delivers the best improvements with balanced binary trees constructed without any linguistic knowledge; this model outperforms standard seq2seq models by up to 2.1 BLEU points, and other methods for incorporating target-side syntax by up to 0.7 BLEU.
Code (1)
Tasks
DecoderMachine TranslationNMTOpen-Ended Question AnsweringTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder
Most neural machine translation (NMT) models are based on the sequential encoder-decoder framework, which makes no use of syntactic information. In this paper, we improve this model by explicitly incorporating source-sid…
DecoderMachine TranslationNMTTranslationCharacter-based Decoding in Tree-to-Sequence Attention-based Neural Machine Translation
This paper reports our systems (UT-AKY) submitted in the 3rd Workshop of Asian Translation 2016 (WAT{'}16) and their results in the English-to-Japanese translation task. Our model is based on the tree-to-sequence Attenti…
DecoderMachine TranslationNMTTranslationTop-down string-to-dependency Neural Machine Translation
Most of modern neural machine translation (NMT) models are based on an encoder-decoder framework with an attention mechanism. While they perform well on standard datasets, they can have trouble in translation of long inp…
Machine TranslationImproving Sequence to Sequence Neural Machine Translation by Utilizing Syntactic Dependency Information
Sequence to Sequence Neural Machine Translation has achieved significant performance in recent years. Yet, there are some existing issues that Neural Machine Translation still does not solve completely. Two of them are t…
DecoderMachine TranslationTranslationInducing Grammars with and for Neural Machine Translation
Machine translation systems require semantic knowledge and grammatical understanding. Neural machine translation (NMT) systems often assume this information is captured by an attention mechanism and a decoder that ensure…
DecoderMachine TranslationNMTTranslation