Debiasing Word Embeddings Improves Multimodal Machine Translation
In recent years, pretrained word embeddings have proved useful for multimodal neural machine translation (NMT) models to address the shortage of available datasets. However, the integration of pretrained word embeddings has not yet been explored extensively. Further, pretrained word embeddings in high dimensional spaces have been reported to suffer from the hubness problem. Although some debiasing techniques have been proposed to address this problem for other natural language processing tasks, they have seldom been studied for multimodal NMT models. In this study, we examine various kinds of word embeddings and introduce two debiasing techniques for three multimodal NMT models and two language pairs -- English-German translation and English-French translation. With our optimal settings, the overall performance of multimodal models was improved by up to +1.93 BLEU and +2.02 METEOR for English-German translation and +1.73 BLEU and +0.95 METEOR for English-French translation.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationMultimodal Machine TranslationNMTTranslationWord EmbeddingsSimilar Papers 제목 키워드 기반
Joint Multiclass Debiasing of Word Embeddings
Bias in Word Embeddings has been a subject of recent interest, along with efforts for its reduction. Current approaches show promising progress towards debiasing single bias dimensions such as gender or race. In this pap…
Word EmbeddingsBlack is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings
Online texts -- across genres, registers, domains, and styles -- are riddled with human stereotypes, expressed in overt or subtle ways. Word embeddings, trained on these texts, perpetuate and amplify these stereotypes, a…
BIG-bench Machine LearningWord EmbeddingsConceptor Debiasing of Word Representations Evaluated on WEAT
Bias in word embeddings such as Word2Vec has been widely investigated, and many efforts made to remove such bias. We show how to use conceptors debiasing to post-process both traditional and contextualized word embedding…
Word EmbeddingsApplying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation
Most works on gender bias focus on intrinsic bias -- removing traces of information about a protected group from the model's internal representation. However, these works are often disconnected from the impact of such de…
Machine TranslationThe Role of Protected Class Word Lists in Bias Identification of Contextualized Word Representations
Systemic bias in word embeddings has been widely reported and studied, and efforts made to debias them; however, new contextualized embeddings such as ELMo and BERT are only now being similarly studied. Standard debiasin…
Word Embeddings