Robust Unsupervised Cross-Lingual Word Embedding using Domain Flow Interpolation
This paper investigates an unsupervised approach towards deriving a universal, cross-lingual word embedding space, where words with similar semantics from different languages are close to one another. Previous adversarial approaches have shown promising results in inducing cross-lingual word embedding without parallel data. However, the training stage shows instability for distant language pairs. Instead of mapping the source language space directly to the target language space, we propose to make use of a sequence of intermediate spaces for smooth bridging. Each intermediate space may be conceived as a pseudo-language space and is introduced via simple linear interpolation. This approach is modeled after domain flow in computer vision, but with a modified objective function. Experiments on intrinsic Bilingual Dictionary Induction tasks show that the proposed approach can improve the robustness of adversarial models with comparable and even better precision. Further experiments on the downstream task of Cross-Lingual Natural Language Inference show that the proposed model achieves significant performance improvement for distant language pairs in downstream tasks compared to state-of-the-art adversarial and non-adversarial models.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual Natural Language InferenceNatural Language InferenceSimilar Papers 제목 키워드 기반
Unsupervised Cross-lingual Word Embedding by Multilingual Neural Language Models
We propose an unsupervised method to obtain cross-lingual embeddings without any parallel data or pre-trained word embeddings. The proposed model, which we call multilingual neural language models, takes sentences of mul…
Cross-Lingual Word EmbeddingsSentenceWord AlignmentWord EmbeddingsUnsupervised Multilingual Word Embedding with Limited Resources using Neural Language Models
Recently, a variety of unsupervised methods have been proposed that map pre-trained word embeddings of different languages into the same space without any parallel data. These methods aim to find a linear transformation …
Word AlignmentWord EmbeddingsUnsupervised Alignment of Distributional Word Embeddings
Cross-domain alignment play a key roles in tasks ranging from machine translation to transfer learning. Recently, purely unsupervised methods operating on monolingual embeddings have successfully been used to infer a bil…
Bilingual Lexicon InductionMachine TranslationStochastic OptimizationTransfer Learning+3Unsupervised Cross-Lingual Information Retrieval using Monolingual Data Only
We propose a fully unsupervised framework for ad-hoc cross-lingual information retrieval (CLIR) which requires no bilingual data at all. The framework leverages shared cross-lingual word embedding spaces in which terms, …
Cross-Lingual Information RetrievalInformation RetrievalRetrievalLearning aligned embeddings for semi-supervised word translation using Maximum Mean Discrepancy
Word translation is an integral part of language translation. In machine translation, each language is considered a domain with its own word embedding. The alignment between word embeddings allows linking semantically eq…
Machine TranslationSentenceTranslationWord Alignment+2