Measuring Machine Translation Errors in New Domains
We develop two techniques for analyzing the effect of porting a machine translation system to a new domain. One is a macro-level analysis that measures how domain shift affects corpus-level evaluation; the second is a micro-level analysis for word-level errors. We apply these methods to understand what happens when a Parliament-trained phrase-based machine translation system is applied in four very different domains: news, medical texts, scientific articles and movie subtitles. We present quantitative and qualitative experiments that highlight opportunities for future research in domain adaptation for machine translation.
Code (0)
등록된 구현이 없습니다.
Tasks
ArticlesDomain AdaptationMachine TranslationTranslationSimilar Papers 제목 키워드 기반
Measuring the Impact of Spelling Errors on the Quality of Machine Translation
In this paper we show how different types of spelling errors influence the quality of machine translation. We also propose a method to evaluate the impact of spelling errors correction on translation quality without expe…
Machine TranslationSentiment AnalysisTranslationPrompting Large Language Models with Human Error Markings for Self-Correcting Machine Translation
While large language models (LLMs) pre-trained on massive amounts of unpaired language data have reached the state-of-the-art in machine translation (MT) of general domain texts, post-editing (PE) is still required to co…
Machine TranslationTranslationBLEU Evaluation of Machine-Translated English-Croatian Legislation
This paper presents work on the evaluation of online available machine translation (MT) service, i.e. Google Translate, for English-Croatian language pair in the domain of legislation. The total set of 200 sentences, for…
Machine TranslationTranslationOn the Relationship between Neural Machine Translation and Word Alignment
Prior researches suggest that attentional neural machine translation (NMT) is able to capture word alignment by attention, however, to our surprise, it almost fails for NMT models with multiple attentional layers except …
Machine TranslationNMTTranslationWord AlignmentUsing Contextual Information for Machine Translation Evaluation
Automatic evaluation of Machine Translation (MT) is typically approached by measuring similarity between the candidate MT and a human reference translation. An important limitation of existing evaluation systems is that …
Machine TranslationSentenceTranslation