Multi-Granularity Optimization for Non-Autoregressive Translation
Despite low latency, non-autoregressive machine translation (NAT) suffers severe performance deterioration due to the naive independence assumption. This assumption is further strengthened by cross-entropy loss, which encourages a strict match between the hypothesis and the reference token by token. To alleviate this issue, we propose multi-granularity optimization for NAT, which collects model behaviors on translation segments of various granularities and integrates feedback for backpropagation. Experiments on four WMT benchmarks show that the proposed method significantly outperforms the baseline models trained with cross-entropy loss, and achieves the best performance on WMT'16 En-Ro and highly competitive results on WMT'14 En-De for fully non-autoregressive translation.
Code (1)
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Progressive Multi-Granularity Training for Non-Autoregressive Translation
Non-autoregressive translation (NAT) significantly accelerates the inference process via predicting the entire target sequence. However, recent studies show that NAT is weak at learning high-mode of knowledge such as one…
SentenceTranslationKnowledge Graph Enhanced Neural Machine Translation via Multi-task Learning on Sub-entity Granularity
Previous studies combining knowledge graph (KG) with neural machine translation (NMT) have two problems: i) Knowledge under-utilization: they only focus on the entities that appear in both KG and training sentence pairs,…
Machine TranslationMulti-Task LearningNMTSentence+1Combining Character and Word Information in Neural Machine Translation Using a Multi-Level Attention
Natural language sentences, being hierarchical, can be represented at different levels of granularity, like words, subwords, or characters. But most neural machine translation systems require the sentence to be represent…
DecoderMachine TranslationSentenceTranslationHint-based Training for Non-Autoregressive Translation
Machine translation is an important real-world application, and neural network-based AutoRegressive Translation (ART) models have achieved very promising accuracy. Due to the unparallelizable nature of the autoregressive…
de-enMachine TranslationTranslationMultilingual Non-Autoregressive Machine Translation without Knowledge Distillation
Multilingual neural machine translation (MNMT) aims at using one single model for multiple translation directions. Recent work applies non-autoregressive Transformers to improve the efficiency of MNMT, but requires expen…
Knowledge DistillationMachine TranslationTranslation