The Effect of Translationese in Machine Translation Test Sets
The effect of translationese has been studied in the field of machine translation (MT), mostly with respect to training data. We study in depth the effect of translationese on test data, using the test sets from the last three editions of WMT's news shared task, containing 17 translation directions. We show evidence that (i) the use of translationese in test sets results in inflated human evaluation scores for MT systems; (ii) in some cases system rankings do change and (iii) the impact translationese has on a translation direction is inversely correlated to the translation quality attainable by state-of-the-art MT systems for that direction.
Code (1)
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Statistical Power and Translationese in Machine Translation Evaluation
The term translationese has been used to describe features of translated text, and in this paper, we provide detailed analysis of potential adverse effects of translationese on machine translation evaluation. Our analysi…
Machine TranslationTranslationOriginal or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance
Human-translated text displays distinct features from naturally written text in the same language. This phenomena, known as translationese, has been argued to confound the machine translation (MT) evaluation. Yet, we fin…
Machine TranslationTranslationOriginal or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance
Human-translated text displays distinct features from naturally written text in the same language. This phenomena, known as translationese, has been argued to confound the machine translation (MT) evaluation. Yet, we fin…
Machine TranslationTranslationUnderstanding Translationese in Cross-Lingual Summarization
Given a document in a source language, cross-lingual summarization (CLS) aims at generating a concise summary in a different target language. Unlike monolingual summarization (MS), naturally occurring source-language doc…
Towards Debiasing Translation Artifacts
Cross-lingual natural language processing relies on translation, either by humans or machines, at different levels, from translating training data to translating test sets. However, compared to original texts in the same…
Natural Language InferenceSentenceTranslation