Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents
Document-level neural machine translation (DocNMT) achieves coherent translations by incorporating cross-sentence context. However, for most language pairs there's a shortage of parallel documents, although parallel sentences are readily available. In this paper, we study whether and how contextual modeling in DocNMT is transferable via multilingual modeling. We focus on the scenario of zero-shot transfer from teacher languages with document level data to student languages with no documents but sentence level data, and for the first time treat document-level translation as a transfer learning problem. Using simple concatenation-based DocNMT, we explore the effect of 3 factors on the transfer: the number of teacher languages with document level data, the balance between document and sentence level data at training, and the data type of document level data (genuine vs. back-translated). Our experiments on Europarl-7 and IWSLT-10 show the feasibility of multilingual transfer for DocNMT, particularly on document-specific metrics. We observe that more teacher languages and adequate data schedule both contribute to better transfer quality. Surprisingly, the transfer is less sensitive to the data type, where multilingual DocNMT delivers decent performance with either back-translated or genuine document pairs.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSentenceTransfer LearningTranslationSimilar Papers 제목 키워드 기반
Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents
Document-level neural machine translation (DocNMT) achieves coherent translations by incorporating cross-sentence context. However, for most language pairs there's a shortage of parallel documents, although parallel sent…
Machine TranslationSentenceTransfer LearningTranslationGoing beyond zero-shot MT: combining phonological, morphological and semantic factors. The UdS-DFKI System at IWSLT 2017
This paper describes the UdS-DFKI participation to the multilingual task of the IWSLT Evaluation 2017. Our approach is based on factored multilingual neural translation systems following the small data and zero-shot trai…
TranslationMultilingual Contextualization of Large Language Models for Document-Level Machine Translation
Large language models (LLMs) have demonstrated strong performance in sentence-level machine translation, but scaling to document-level translation remains challenging, particularly in modeling long-range dependencies and…
Document Level Machine TranslationDocument TranslationMachine TranslationSentence+1Improving Multilingual Translation by Representation and Gradient Regularization
Multilingual Neural Machine Translation (NMT) enables one model to serve all translation directions, including ones that are unseen during training, i.e. zero-shot translation. Despite being theoretically attractive, cur…
DecoderMachine TranslationNMTSentence+1Advancing Multilingual Pre-training: TRIP Triangular Document-level Pre-training for Multilingual Language Models
Despite the success of multilingual sequence-to-sequence pre-training, most existing approaches rely on document-level monolingual corpora in many different languages, sentence-level bilingual corpora,\footnote{In this p…
Abstractive Text SummarizationCross-Lingual Abstractive SummarizationDocument Level Machine TranslationMachine Translation+2