Analyzing the Limitations of Cross-lingual Word Embedding Mappings
Recent research in cross-lingual word embeddings has almost exclusively focused on offline methods, which independently train word embeddings in different languages and map them to a shared space through linear transformations. While several authors have questioned the underlying isomorphism assumption, which states that word embeddings in different languages have approximately the same structure, it is not clear whether this is an inherent limitation of mapping approaches or a more general issue when learning cross-lingual embeddings. So as to answer this question, we experiment with parallel corpora, which allows us to compare offline mapping to an extension of skip-gram that jointly learns both embedding spaces. We observe that, under these ideal conditions, joint learning yields to more isomorphic embeddings, is less sensitive to hubness, and obtains stronger results in bilingual lexicon induction. We thus conclude that current mapping methods do have strong limitations, calling for further research to jointly learn cross-lingual embeddings with a weaker cross-lingual signal.
Code (0)
등록된 구현이 없습니다.
Tasks
Bilingual Lexicon InductionCross-Lingual Word EmbeddingsWord EmbeddingsSimilar Papers 제목 키워드 기반
On the Robustness of Unsupervised and Semi-supervised Cross-lingual Word Embedding Learning
Cross-lingual word embeddings are vector representations of words in different languages where words with similar meaning are represented by similar vectors, regardless of the language. Recent developments which construc…
Cross-Lingual Word EmbeddingsWord EmbeddingsImproving Unsupervised Word-by-Word Translation with Language Model and Denoising Autoencoder
Unsupervised learning of cross-lingual word embedding offers elegant matching of words across languages, but has fundamental limitations in translating sentences. In this paper, we propose simple yet effective methods to…
DenoisingLanguage ModelingLanguage ModellingTranslation+1What's in an Embedding? Analyzing Word Embeddings through Multilingual Evaluation
Cross-lingual Word Embeddings beyond Zero-shot Machine Translation
We explore the transferability of a multilingual neural machine translation model to unseen languages when the transfer is grounded solely on the cross-lingual word embeddings. Our experimental results show that the tran…
Cross-Lingual Word EmbeddingsMachine TranslationTranslationWord Embeddings+1Multi-Adversarial Learning for Cross-Lingual Word Embeddings
Generative adversarial networks (GANs) have succeeded in inducing cross-lingual word embeddings -- maps of matching words across languages -- without supervision. Despite these successes, GANs' performance for the diffic…
Bilingual Lexicon InductionCross-Lingual Word EmbeddingsWord Embeddings