Grounded Word Sense Translation
Recent work on visually grounded language learning has focused on broader applications of grounded representations, such as visual question answering and multimodal machine translation. In this paper we consider grounded word sense translation, i.e. the task of correctly translating an ambiguous source word given the corresponding textual and visual context. Our main objective is to investigate the extent to which images help improve word-level (lexical) translation quality. We do so by first studying the dataset for this task to understand the scope and challenges of the task. We then explore different data settings, image features, and ways of grounding to investigate the gain from using images in each of the combinations. We find that grounding on the image is specially beneficial in weaker unidirectional recurrent translation models. We observe that adding structured image information leads to stronger gains in lexical translation accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Grounded language learningMachine TranslationMultimodal Machine TranslationQuestion AnsweringTranslationVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
UAlberta at SemEval-2021 Task 2: Determining Sense Synonymy via Translations
We describe the University of Alberta systems for the SemEval-2021 Word-in-Context (WiC) disambiguation task. We explore the use of translation information for deciding whether two different tokens of the same word corre…
Explainable ModelsTask 2TranslationWord Sense-Aware Machine Translation: Including Senses as Contextual Features for Improved Translation Models
Although it is commonly assumed that word sense disambiguation (WSD) should help to improve lexical choice and improve the quality of machine translation systems, how to successfully integrate word senses into such syste…
Machine TranslationTranslationWord Sense DisambiguationImproving Machine Translation of Rare and Unseen Word Senses
The performance of NMT systems has improved drastically in the past few years but the translation of multi-sense words still poses a challenge. Since word senses are not represented uniformly in the parallel corpora used…
Bilingual Lexicon InductionMachine TranslationNMTTranslation+2Improving Translation Selection with Supersenses
Selecting appropriate translations for source words with multiple meanings still remains a challenge for statistical machine translation (SMT). One reason for this is that most SMT systems are not good at detecting the p…
Machine TranslationTranslationWord Sense DisambiguationOne Sense per Translation
Word sense disambiguation (WSD) is the task of determining the sense of a word in context. Translations have been used in WSD as a source of knowledge, and even as a means of delimiting word senses. In this paper, we def…
TranslationWord Sense Disambiguation