Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance
Human-translated text displays distinct features from naturally written text in the same language. This phenomena, known as translationese, has been argued to confound the machine translation (MT) evaluation. Yet, we find that existing work on translationese neglects some important factors and the conclusions are mostly correlational but not causal. In this work, we collect CausalMT, a dataset where the MT training data are also labeled with the human translation directions. We inspect two critical factors, the train-test alignment (whether the human translation directions in the training and test sets are aligned), and data-model alignment (whether the model learns in the same direction as the human translation direction in the dataset). We show that these two factors have a large causal effect on the MT performance, in addition to the test-model misalignment highlighted by existing work on the impact of translationese in the test set. In light of our findings, we provide a set of suggestions for MT training and evaluation.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance
Human-translated text displays distinct features from naturally written text in the same language. This phenomena, known as translationese, has been argued to confound the machine translation (MT) evaluation. Yet, we fin…
Machine TranslationTranslationTranslating away Translationese without Parallel Data
Translated texts exhibit systematic linguistic differences compared to original texts in the same language, and these differences are referred to as translationese. Translationese has effects on various cross-lingual nat…
Binary ClassificationLanguage ModellingSemantic SimilaritySemantic Textual Similarity+1Lost in Translationese? Reducing Translation Effect Using Abstract Meaning Representation
Translated texts bear several hallmarks distinct from texts originating in the language. Though individual translated texts are often fluent and preserve meaning, at a large scale, translated texts have statistical tende…
Abstract Meaning RepresentationMachine TranslationParaphrase GenerationTranslationOriginal or Translated? On the Use of Parallel Data for Translation Quality Estimation
Machine Translation Quality Estimation (QE) is the task of evaluating translation output in the absence of human-written references. Due to the scarcity of human-labeled QE data, previous works attempted to utilize the a…
Data AugmentationMachine TranslationSentenceTranslationDialects of Translationese Shape Language Model Learning
Machine-translated data is widely used in multilingual NLP, particularly where native text is scarce. However, translated text differs systematically from native text. This phenomenon is known as translationese, and it r…
Linguistic AcceptabilityLanguage Modelling