Evaluating Sub-word Embeddings in Cross-lingual Models
Cross-lingual word embeddings create a shared space for embeddings in two languages, and enable knowledge to be transferred between languages for tasks such as bilingual lexicon induction. One problem, however, is out-of-vocabulary (OOV) words, for which no embeddings are available. This is particularly problematic for low-resource and morphologically-rich languages, which often have relatively high OOV rates. Approaches to learning sub-word embeddings have been proposed to address the problem of OOV words, but most prior work has not considered sub-word embeddings in cross-lingual models. In this paper, we consider whether sub-word embeddings can be leveraged to form cross-lingual embeddings for OOV words. Specifically, we consider a novel bilingual lexicon induction task focused on OOV words, for language pairs covering several language families. Our results indicate that cross-lingual representations for OOV words can indeed be formed from sub-word embeddings, including in the case of a truly low-resource morphologically-rich language.
Code (0)
등록된 구현이 없습니다.
Tasks
Bilingual Lexicon InductionCross-Lingual Word EmbeddingsWord EmbeddingsSimilar Papers 제목 키워드 기반
Evaluating a Joint Training Approach for Learning Cross-lingual Embeddings with Sub-word Information without Parallel Corpora on Lower-resource Languages
Cross-lingual word embeddings provide a way for information to be transferred between languages. In this paper we evaluate an extension of a joint training approach to learning cross-lingual embeddings that incorporates …
Bilingual Lexicon InductionCross-Lingual Word EmbeddingsWord EmbeddingsCLUSE: Cross-Lingual Unsupervised Sense Embeddings
This paper proposes a modularized sense induction and representation learning model that jointly learns bilingual sense embeddings that align well in the vector space, where the cross-lingual signal in the English-Chines…
Representation LearningWord SimilarityEvaluating bilingual word embeddings on the long tail
Bilingual word embeddings are useful for bilingual lexicon induction, the task of mining translations of given words. Many studies have shown that bilingual word embeddings perform well for bilingual lexicon induction bu…
Bilingual Lexicon InductionMachine TranslationWord EmbeddingsWasserstein distances for evaluating cross-lingual embeddings
Word embeddings are high dimensional vector representations of words that capture their semantic similarity in the vector space. There exist several algorithms for learning such embeddings both for a single language as w…
Cross-Lingual Document ClassificationDocument ClassificationRetrievalSemantic Similarity+2Evaluating Word Embeddings for Indonesian--English Code-Mixed Text Based on Synthetic Data
Code-mixed texts are abundant, especially in social media, and poses a problem for NLP tools, which are typically trained on monolingual corpora. In this paper, we explore and evaluate different types of word embeddings …
Sentiment AnalysisWord Embeddings