Modeling Coverage for Non-Autoregressive Neural Machine Translation
Non-Autoregressive Neural Machine Translation (NAT) has achieved significant inference speedup by generating all tokens simultaneously. Despite its high efficiency, NAT usually suffers from two kinds of translation errors: over-translation (e.g. repeated tokens) and under-translation (e.g. missing translations), which eventually limits the translation quality. In this paper, we argue that these issues of NAT can be addressed through coverage modeling, which has been proved to be useful in autoregressive decoding. We propose a novel Coverage-NAT to model the coverage information directly by a token-level coverage iterative refinement mechanism and a sentence-level coverage agreement, which can remind the model if a source token has been translated or not and improve the semantics consistency between the translation and the source, respectively. Experimental results on WMT14 En-De and WMT16 En-Ro translation tasks show that our method can alleviate those errors and achieve strong improvements over the baseline system.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSentenceTranslationSimilar Papers 제목 키워드 기반
Shared Latent Space by Both Languages in Non-Autoregressive Neural Machine Translation
Non-autoregressive neural machine translation (NAT) offers substantial translation speed up compared to autoregressive neural machine translation (AT) at the cost of translation quality. Latent variable modeling has emer…
Machine TranslationTranslationModeling Coverage for Neural Machine Translation
Attention mechanism has enhanced state-of-the-art Neural Machine Translation (NMT) by jointly learning to align and translate. It tends to ignore past alignment information, however, which often leads to over-translation…
Machine TranslationNMTTranslationNon-Autoregressive Translation by Learning Target Categorical Codes
Non-autoregressive Transformer is a promising text generation model. However, current non-autoregressive models still fall behind their autoregressive counterparts in translation quality. We attribute this accuracy gap t…
AttributeDecoderMachine TranslationText Generation+1The RoyalFlush System for the WMT 2022 Efficiency Task
This paper describes the submission of the RoyalFlush neural machine translation system for the WMT 2022 translation efficiency task. Unlike the commonly used autoregressive translation system, we adopted a two-stage tra…
DecoderGPUKnowledge DistillationMachine Translation+1Aligned Cross Entropy for Non-Autoregressive Machine Translation
Non-autoregressive machine translation models significantly speed up decoding by allowing for parallel prediction of the entire target sequence. However, modeling word order is more challenging due to the lack of autoreg…
Machine TranslationTranslation