Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation
Unsupervised bilingual word embedding (UBWE), together with other technologies such as back-translation and denoising, has helped unsupervised neural machine translation (UNMT) achieve remarkable results in several language pairs. In previous methods, UBWE is first trained using non-parallel monolingual corpora and then this pre-trained UBWE is used to initialize the word embedding in the encoder and decoder of UNMT. That is, the training of UBWE and UNMT are separate. In this paper, we first empirically investigate the relationship between UBWE and UNMT. The empirical findings show that the performance of UNMT is significantly affected by the performance of UBWE. Thus, we propose two methods that train UNMT with UBWE agreement. Empirical results on several language pairs show that the proposed methods significantly outperform conventional UNMT.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderDenoisingMachine TranslationTranslationSimilar Papers 제목 키워드 기반
Unsupervised Joint Training of Bilingual Word Embeddings
State-of-the-art methods for unsupervised bilingual word embeddings (BWE) train a mapping function that maps pre-trained monolingual word embeddings into a bilingual space. Despite its remarkable results, unsupervised ma…
Machine TranslationTranslationUnsupervised Machine TranslationWord EmbeddingsOn the Limitations of Unsupervised Bilingual Dictionary Induction
Unsupervised machine translation---i.e., not assuming any cross-lingual supervision signal, whether a dictionary, translations, or comparable corpora---seems impossible, but nevertheless, Lample et al. (2018) recently pr…
Graph SimilarityMachine TranslationTranslationUnsupervised Machine TranslationUnsupervised Word Mapping Using Structural Similarities in Monolingual Embeddings
Most existing methods for automatic bilingual dictionary induction rely on prior alignments between the source and target languages, such as parallel corpora or seed dictionaries. For many language pairs, such supervised…
Word EmbeddingsLearning Bilingual Sentiment-Specific Word Embeddings without Cross-lingual Supervision
Word embeddings learned in two languages can be mapped to a common space to produce Bilingual Word Embeddings (BWE). Unsupervised BWE methods learn such a mapping without any parallel data. However, these methods are mai…
Sentiment AnalysisTranslationWord EmbeddingsWord Similarity+1A Graph-based Coarse-to-fine Method for Unsupervised Bilingual Lexicon Induction
Unsupervised bilingual lexicon induction is the task of inducing word translations from monolingual corpora of two languages. Recent methods are mostly based on unsupervised cross-lingual word embeddings, the key to whic…
Bilingual Lexicon InductionCross-Lingual Word EmbeddingsWord EmbeddingsWord Translation