PreQuEL: Quality Estimation of Machine Translation Outputs in Advance
We present the task of PreQuEL, Pre-(Quality-Estimation) Learning. A PreQuEL system predicts how well a given sentence will be translated, without recourse to the actual translation, thus eschewing unnecessary resource allocation when translation quality is bound to be low. PreQuEL can be defined relative to a given MT system (e.g., some industry service) or generally relative to the state-of-the-art. From a theoretical perspective, PreQuEL places the focus on the source text, tracing properties, possibly linguistic features, that make a sentence harder to machine translate. We develop a baseline model for the task and analyze its performance. We also develop a data augmentation method (from parallel corpora), that improves results substantially. We show that this augmentation method can improve the performance of the Quality-Estimation task as well. We investigate the properties of the input text that our model is sensitive to, by testing it on challenge sets and different languages. We conclude that it is aware of syntactic and semantic distinctions, and correlates and even over-emphasizes the importance of standard NLP features.
Code (1)
Tasks
Data AugmentationMachine TranslationSentenceTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errors
A major challenge in the practical use of Machine Translation (MT) is that users lack guidance to make informed decisions about when to rely on outputs. Progress in quality estimation research provides techniques to auto…
Decision MakingMachine TranslationQuality Estimation of Machine Translated Texts based on Direct Evidence from Training Data
Current Machine Translation systems achieve very good results on a growing variety of language pairs and data sets. However, it is now well known that they produce fluent translation outputs that often can contain import…
Machine TranslationTranslationJapanese to English/Chinese/Korean Datasets for Translation Quality Estimation and Automatic Post-Editing
Aiming at facilitating the research on quality estimation (QE) and automatic post-editing (APE) of machine translation (MT) outputs, especially for those among Asian languages, we have created new datasets for Japanese t…
Automatic Post-EditingBenchmarkingMachine TranslationSentence+1Comparative Quality Estimation: Automatic Sentence-Level Ranking of Multiple Machine Translation Outputs
Ensemble-based Transfer Learning for Low-resource Machine Translation Quality Estimation
Quality Estimation (QE) of Machine Translation (MT) is a task to estimate the quality scores for given translation outputs from an unknown MT system. However, QE scores for low-resource languages are usually intractable …
Machine TranslationMiscellaneousSentenceTransfer Learning+1