Non-Autoregressive Document-Level Machine Translation
Non-autoregressive translation (NAT) models achieve comparable performance and superior speed compared to auto-regressive translation (AT) models in the context of sentence-level machine translation (MT). However, their abilities are unexplored in document-level MT, hindering their usage in real scenarios. In this paper, we conduct a comprehensive examination of typical NAT models in the context of document-level MT and further propose a simple but effective design of sentence alignment between source and target. Experiments show that NAT models achieve high acceleration on documents, and sentence alignment significantly enhances their performance. However, current NAT models still have a significant performance gap compared to their AT counterparts. Further investigation reveals that NAT models suffer more from the multi-modality and misalignment issues in the context of document-level MT, and current NAT models struggle with exploiting document context and handling discourse phenomena. We delve into these challenges and provide our code at \url{https://github.com/baoguangsheng/nat-on-doc}.
Code (1)
Tasks
Document Level Machine TranslationMachine TranslationSentenceText GenerationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Better Document-Level Machine Translation with Bayes' Rule
We show that Bayes' rule provides an effective mechanism for creating document translation models that can be learned from only parallel sentences and monolingual documents---a compelling benefit as parallel documents ar…
Document Level Machine TranslationDocument TranslationLanguage ModelingLanguage Modelling+3Tencent AI Lab Machine Translation Systems for WMT20 Chat Translation Task
This paper describes the Tencent AI Lab’s submission of the WMT 2020 shared task on chat translation in English-German. Our neural machine translation (NMT) systems are built on sentence-level, document-level, non-autore…
General KnowledgeMachine TranslationNMTSentence+1Document Level Hierarchical Transformer
Generating long and coherent text is an important and challenging task encompassing many application areas such as summarization, document level machine translation and story generation. Despite the success in modeling i…
Document Level Machine TranslationImitation LearningMachine TranslationSentence+2Using Perturbed Length-aware Positional Encoding for Non-autoregressive Neural Machine Translation
Non-autoregressive neural machine translation (NAT) usually employs sequence-level knowledge distillation using autoregressive neural machine translation (AT) as its teacher model. However, a NAT model often outputs shor…
Knowledge DistillationMachine TranslationTranslationDocument-Level Neural Machine Translation Using BERT as Context Encoder
Large-scale pre-trained representations such as BERT have been widely used in many natural language understanding tasks. The methods of incorporating BERT into document-level machine translation are still being explored.…
DecoderDocument Level Machine TranslationMachine TranslationNatural Language Understanding+2