The MUCOW word sense disambiguation test suite at WMT 2020
This paper reports on our participation with the MUCOW test suite at the WMT 2020 news translation task. We introduced MUCOW at WMT 2019 to measure the ability of MT systems to perform word sense disambiguation (WSD), i.e., to translate an ambiguous word with its correct sense. MUCOW is created automatically using existing resources, and the evaluation process is also entirely automated. We evaluate all participating systems of the language pairs English -> Czech, English -> German, and English -> Russian and compare the results with those obtained at WMT 2019. While current NMT systems are fairly good at handling ambiguous source words, we could not identify any substantial progress - at least to the extent that it is measurable by the MUCOW method - in that area over the last year.
Code (1)
Tasks
NMTTranslationWord Sense DisambiguationSimilar Papers 제목 키워드 기반
The MuCoW Test Suite at WMT 2019: Automatically Harvested Multilingual Contrastive Word Sense Disambiguation Test Sets for Machine Translation
Supervised Neural Machine Translation (NMT) systems currently achieve impressive translation quality for many language pairs. One of the key features of a correct translation is the ability to perform word sense disambig…
Machine TranslationNMTSentenceTranslation+1An Evaluation Benchmark for Testing the Word Sense Disambiguation Capabilities of Machine Translation Systems
Lexical ambiguity is one of the many challenging linguistic phenomena involved in translation, i.e., translating an ambiguous word with its correct sense. In this respect, previous work has shown that the translation qua…
Machine TranslationTranslationWord Sense DisambiguationImproving Machine Translation of Rare and Unseen Word Senses
The performance of NMT systems has improved drastically in the past few years but the translation of multi-sense words still poses a challenge. Since word senses are not represented uniformly in the parallel corpora used…
Bilingual Lexicon InductionMachine TranslationNMTTranslation+2Improving Word Sense Disambiguation in Neural Machine Translation with Salient Document Context
Lexical ambiguity is a challenging and pervasive problem in machine translation (\mt). We introduce a simple and scalable approach to resolve translation ambiguity by incorporating a small amount of extra-sentential cont…
Machine TranslationSentenceTranslationWord Sense DisambiguationGame Theory Meets Embeddings: a Unified Framework for Word Sense Disambiguation
Game-theoretic models, thanks to their intrinsic ability to exploit contextual information, have shown to be particularly suited for the Word Sense Disambiguation task. They represent ambiguous words as the players of a …
Word Sense Disambiguation