Depth Growing for Neural Machine Translation
While very deep neural networks have shown effectiveness for computer vision and text classification applications, how to increase the network depth of neural machine translation (NMT) models for better translation quality remains a challenging problem. Directly stacking more blocks to the NMT model results in no improvement and even reduces performance. In this work, we propose an effective two-stage approach with three specially designed components to construct deeper NMT models, which result in significant improvements over the strong Transformer baselines on WMT$14$ English$\to$German and English$\to$French translation tasks\footnote{Our code is available at \url{https://github.com/apeterswu/Depth_Growing_NMT}}.
Code (1)
Tasks
Machine TranslationNMTtext-classificationText ClassificationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Deep Architectures for Neural Machine Translation
It has been shown that increasing model depth improves the quality of neural machine translation. However, different architectural variants to increase model depth have been proposed, and so far, there has been no thorou…
DecoderGPUMachine TranslationTranslationEED: Extended Edit Distance Measure for Machine Translation
Over the years a number of machine translation metrics have been developed in order to evaluate the accuracy and quality of machine-generated translations. Metrics such as BLEU and TER have been used for decades. However…
Machine TranslationTranslationTraining Flexible Depth Model by Multi-Task Learning for Neural Machine Translation
The standard neural machine translation model can only decode with the same depth configuration as training. Restricted by this feature, we have to deploy models of various sizes to maintain the same translation latency,…
Machine TranslationMulti-Task LearningTranslationNeural Network Based Bilingual Language Model Growing for Statistical Machine Translation
Incorporating Human Translator Style into English-Turkish Literary Machine Translation
Although machine translation systems are mostly designed to serve in the general domain, there is a growing tendency to adapt these systems to other domains like literary translation. In this paper, we focus on English-T…
Data AugmentationMachine TranslationTranslation