Can Statistical Post-Editing with a Small Parallel Corpus Save a Weak MT Engine?
Statistical post-editing has been shown in several studies to increase BLEU score for rule-based MT systems. However, previous studies have relied solely on BLEU and have not conducted further study to determine whether those gains indicated an increase in quality or in score alone. In this work we conduct a human evaluation of statistical post-edited output from a weak rule-based MT system, comparing the results with the output of the original rule-based system and a phrase-based statistical MT system trained on the same data. We show that for this weak rule-based system, despite significant BLEU score increases, human evaluators prefer the output of the original system. While this is not a generally conclusive condemnation of statistical post-editing, this result does cast doubt on the efficacy of statistical post-editing for weak MT systems and on the reliability of BLEU score for comparison between weak rule-based and hybrid systems built from them.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSimilar Papers 제목 키워드 기반
iMAG : MT-postediting, translation quality evaluation and parallel corpus production (iMAG : post-\'edition, \'evaluation de qualit\'e de TA et production d'un corpus parall\`ele) [in French]
Demo of iMAG Possibilities: MT-postediting, Translation Quality Evaluation, Parallel Corpus Production
Simple Automatic Post-editing for Arabic-Japanese Machine Translation
A common bottleneck for developing machine translation (MT) systems for some language pairs is the lack of direct parallel translation data sets, in general and in certain domains. Alternative solutions such as zero-shot…
ArticlesAutomatic Post-EditingMachine TranslationTranslationMulti-Engine and Multi-Alignment Based Automatic Post-Editing and its Impact on Translation Productivity
In this paper we combine two strands of machine translation (MT) research: automatic post-editing (APE) and multi-engine (system combination) MT. APE systems learn a target-language-side second stage MT system from the d…
Automatic Post-EditingMachine TranslationTranslationCan Automatic Post-Editing Improve NMT?
Automatic post-editing (APE) aims to improve machine translations, thereby reducing human post-editing effort. APE has had notable success when used with statistical machine translation (SMT) systems but has not been as …
Automatic Post-EditingMachine TranslationNMTSpecificity+1