Revisiting Round-Trip Translation for Quality Estimation
Quality estimation (QE) is the task of automatically evaluating the quality of translations without human-translated references. Calculating BLEU between the input sentence and round-trip translation (RTT) was once considered as a metric for QE, however, it was found to be a poor predictor of translation quality. Recently, various pre-trained language models have made breakthroughs in NLP tasks by providing semantically meaningful word and sentence embeddings. In this paper, we employ semantic embeddings to RTT-based QE. Our method achieves the highest correlations with human judgments, compared to previous WMT 2019 quality estimation metric task submissions. While backward translation models can be a drawback when using RTT, we observe that with semantic-level metrics, RTT-based QE is robust to the choice of the backward translation system. Additionally, the proposed method shows consistent performance for both SMT and NMT forward translation systems, implying the method does not penalize a certain type of model.
Code (0)
등록된 구현이 없습니다.
Tasks
NMTSentenceSentence EmbeddingsTranslationSimilar Papers 제목 키워드 기반
Quality Estimation Using Round-trip Translation with Sentence Embeddings
Estimating the quality of machine translation systems has been an ongoing challenge for researchers in this field. Many previous attempts at using round-trip translation as a measure of quality have failed, and there is …
Machine TranslationRepresentation LearningSentenceSentence Embeddings+1Rethinking Round-Trip Translation for Machine Translation Evaluation
Automatic evaluation on low-resource language translation suffers from a deficiency of parallel corpora. Round-trip translation could be served as a clever and straightforward technique to alleviate the requirement of th…
Machine TranslationTranslationThe JHU-Microsoft Submission for WMT21 Quality Estimation Shared Task
This paper presents the JHU-Microsoft joint submission for WMT 2021 quality estimation shared task. We only participate in Task 2 (post-editing effort estimation) of the shared task, focusing on the target-side word-leve…
Data AugmentationTask 2TranslationBilingual Low-Resource Neural Machine Translation with Round-Tripping: The Case of Persian-Spanish
The quality of Neural Machine Translation (NMT), as a data-driven approach, massively depends on quantity, quality, and relevance of the training dataset. Such approaches have achieved promising results for bilingually h…
Low Resource Neural Machine TranslationLow-Resource Neural Machine TranslationLow Resource NMTMachine Translation+2Outbound Translation User Interface Ptakopet: A Pilot Study
It is not uncommon for Internet users to have to produce a text in a foreign language they have very little knowledge of and are unable to verify the translation quality. We call the task "outbound translation" and explo…
Translation