SAO WMT19 Test Suite: Machine Translation of Audit Reports
This paper describes a machine translation test set of documents from the auditing domain and its use as one of the {``}test suites{''} in the WMT19 News Translation Task for translation directions involving Czech, English and German. Our evaluation suggests that current MT systems optimized for the general news domain can perform quite well even in the particular domain of audit reports. The detailed manual evaluation however indicates that deep factual knowledge of the domain is necessary. For the naked eye of a non-expert, translations by many systems seem almost perfect and automatic MT evaluation with one reference is practically useless for considering these details. Furthermore, we show on a sample document from the domain of agreements that even the best systems completely fail in preserving the semantics of the agreement, namely the identity of the parties.
Code (1)
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
SAO WMT19 Test Suite: Machine Translation of Audit Reports
This paper describes a machine translation test set of documents from the auditing domain and its use as one of the "test suites" in the WMT19 News Translation Task for translation directions involving Czech, English and…
Machine TranslationTranslationPredicting Distresses using Deep Learning of Text Segments in Annual Reports
Corporate distress models typically only employ the numerical financial variables in the firms' annual reports. We develop a model that employs the unstructured textual data in the reports as well, namely the auditors' r…
DescriptiveFinding Alternative Translations in a Large Corpus of Movie Subtitle
OpenSubtitles.org provides a large collection of user contributed subtitles in various languages for movies and TV programs. Subtitle translations are valuable resources for cross-lingual studies and machine translation …
Machine TranslationSentenceTranslationThe Box is in the Pen: Evaluating Commonsense Reasoning in Neural Machine Translation
Does neural machine translation yield translations that are congenial with common sense? In this paper, we present a test suite to evaluate the commonsense reasoning capability of neural machine translation. The test sui…
Common Sense ReasoningMachine TranslationSentenceTranslationMachine Translation with parfda, Moses, kenlm, nplm, and PRO
We build parfda Moses statistical machine translation (SMT) models for most language pairs in the news translation task. We experiment with a hybrid approach using neural language models integrated into Moses. We obtain …
Machine TranslationTranslation