Investigating Length Issues in Document-level Machine Translation
Transformer architectures are increasingly effective at processing and generating very long chunks of texts, opening new perspectives for document-level machine translation (MT). In this work, we challenge the ability of MT systems to handle texts comprising up to several thousands of tokens. We design and implement a new approach designed to precisely measure the effect of length increments on MT outputs. Our experiments with two representative architectures unambiguously show that (a)~translation performance decreases with the length of the input text; (b)~the position of sentences within the document matters and translation quality is higher for sentences occurring earlier in a document. We further show that manipulating the distribution of document lengths and of positional embeddings only marginally mitigates such problems. Our results suggest that even though document-level MT is computationally feasible, it does not yet match the performance of sentence-based MT.
Code (0)
등록된 구현이 없습니다.
Tasks
Document Level Machine TranslationMachine TranslationSentenceTranslationSimilar Papers 제목 키워드 기반
Efficiently Exploring Large Language Models for Document-Level Machine Translation with In-context Learning
Large language models (LLMs) exhibit outstanding performance in machine translation via in-context learning. In contrast to sentence-level translation, document-level translation (DOCMT) by LLMs based on in-context learn…
Document Level Machine TranslationIn-Context LearningMachine TranslationSentence+1Extending Automatic Machine Translation Evaluation to Book-Length Documents
Despite Large Language Models (LLMs) demonstrating superior translation performance and long-context capabilities, evaluation methodologies remain constrained to sentence-level assessment due to dataset limitations, toke…
Machine TranslationAddressing the Length Bias Problem in Document-Level Neural Machine Translation
Document-level neural machine translation (DNMT) has shown promising results by incorporating more context information. However, this approach also introduces a length bias problem, whereby DNMT suffers from significant …
Machine TranslationTranslationTowards Document-Level Human MT Evaluation: On the Issues of Annotator Agreement, Effort and Misevaluation
Document-level human evaluation of machine translation (MT) has been raising interest in the community. However, little is known about the issues of using document-level methodologies to assess MT quality. In this articl…
Machine TranslationTranslationHow Much Context Span is Enough? Examining Context-Related Issues for Document-level MT
This paper analyses how much context span is necessary to solve different context-related issues, namely, reference, ellipsis, gender, number, lexical ambiguity, and terminology when translating from English into Portugu…
World Knowledge