Improving Evaluation of English-Czech MT through Paraphrasing

2014-05-01 · LREC 2014 5 · Petra Baran{\v{c}}{\'\i}kov{\'a}, Rudolf Rosa, Ale{\v{s}} Tamchyna

In this paper, we present a method of improving the accuracy of machine translation evaluation of Czech sentences. Given a reference sentence, our algorithm transforms it by targeted paraphrasing into a new synthetic reference sentence that is closer in wording to the machine translation output, but at the same time preserves the meaning of the original reference sentence. Grammatical correctness of the new reference sentence is provided by applying Depfix on newly created paraphrases. Depfix is a system for post-editing English-to-Czech machine translation outputs. We adjusted it to fix the errors in paraphrased sentences. Due to a noisy source of our paraphrases, we experiment with adding word alignment. However, the alignment reduces the number of paraphrases found and the best results were achieved by a simple greedy method with only one-word paraphrases thanks to their intensive filtering. BLEU scores computed using these new reference sentences show significantly higher correlation with human judgment than scores computed on the original reference sentences.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Automatic Post-EditingMachine TranslationSentenceTranslation Word Alignment

Manual and Automatic Paraphrases for MT Evaluation

2016-05-01 · LREC 2016 5 · Ale{\v{s}} Tamchyna, Petra Baran{\v{c}}{\'\i}kov{\'a}

Paraphrasing of reference translations has been shown to improve the correlation with human judgements in automatic evaluation of machine translation (MT) outputs. In this work, we present a new dataset for evaluating En…

Machine Translation Translation

Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing

2020-04-30 · EMNLP 2020 11 · Brian Thompson, Matt Post

We frame the task of machine translation evaluation as one of scoring machine translation output with a sequence-to-sequence paraphraser, conditioned on a human reference. We propose training the paraphraser as a multili…

Machine Translation NMT Translation

ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation

2019-01-11 · J. Edward Hu, Rachel Rudinger, Matt Post, Benjamin Van Durme

We present ParaBank, a large-scale English paraphrase dataset that surpasses prior work in both quantity and quality. Following the approach of ParaNMT, we train a Czech-English neural machine translation (NMT) system to…

Diversity Machine Translation NMT Semantic Similarity+4

The WMT'18 Morpheval test suites for English-Czech, English-German, English-Finnish and Turkish-English

2018-10-01 · WS 2018 10 · Franck Burlot, Yves Scherrer, Vinit Ravishankar, Ond{\v{r}}ej Bojar 외

Progress in the quality of machine translation output calls for new automatic evaluation procedures and metrics. In this paper, we extend the Morpheval protocol introduced by Burlot and Yvon (2017) for the English-to-Cze…

Machine Translation Translation

Mitigating Language Barriers in Education: Developing Multilingual Digital Learning Materials with Machine Translation

2025-09-11 · Lucie Poláková, Martin Popel, Věra Kloudová, Michal Novák 외 arxiv

The EdUKate project combines digital education, linguistics, translation studies, and machine translation to develop multilingual learning materials for Czech primary and secondary schools. Launched through collaboration…

Machine Translation