Linguistic evaluation of German-English Machine Translation using a Test Suite
We present the results of the application of a grammatical test suite for German$\rightarrow$English MT on the systems submitted at WMT19, with a detailed analysis for 107 phenomena organized in 14 categories. The systems still translate wrong one out of four test items in average. Low performance is indicated for idioms, modals, pseudo-clefts, multi-word expressions and verb valency. When compared to last year, there has been a improvement of function words, non-verbal agreement and punctuation. More detailed conclusions about particular systems and phenomena are also presented.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Linguistic Evaluation for the 2021 State-of-the-art Machine Translation Systems for German to English and English to German
We are using a semi-automated test suite in order to provide a fine-grained linguistic evaluation for state-of-the-art machine translation systems. The evaluation includes 18 German to English and 18 English to German sy…
Machine TranslationTranslationObserving the Learning Curve of NMT Systems With Regard to Linguistic Phenomena
In this paper we present our observations and evaluations by observing the linguistic performance of the system on several steps on the training process of various English-to-German Neural Machine Translation models. The…
Machine TranslationNMTTranslationA Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output
This paper presents a fine-grained test suite for the language pair German–English. The test suite is based on a number of linguistically motivated categories and phenomena and the semi-automatic evaluation is carried ou…
Machine TranslationScheduled Multi-Task Learning: From Syntax to Translation
Neural encoder-decoder models of machine translation have achieved impressive results, while learning linguistic knowledge of both the source and target languages in an implicit end-to-end manner. We propose a framework …
DecoderMachine TranslationMulti-Task LearningTranslationLinguistic Evaluation of Support Verb Constructions by OpenLogos and Google Translate
This paper presents a systematic human evaluation of translations of English support verb constructions produced by a rule-based machine translation (RBMT) system (OpenLogos) and a statistical machine translation (SMT) s…
Machine TranslationTranslation