Bilingual Lexicon Induction across Orthographically-distinct Under-Resourced Dravidian Languages
Bilingual lexicons are a vital tool for under-resourced languages and recent state-of-the-art approaches to this leverage pretrained monolingual word embeddings using supervised or semi-supervised approaches. However, these approaches require cross-lingual information such as seed dictionaries to train the model and find a linear transformation between the word embedding spaces. Especially in the case of low-resourced languages, seed dictionaries are not readily available, and as such, these methods produce extremely weak results on these languages. In this work, we focus on the Dravidian languages, namely Tamil, Telugu, Kannada, and Malayalam, which are even more challenging as they are written in unique scripts. To take advantage of orthographic information and cognates in these languages, we bring the related languages into a single script. Previous approaches have used linguistically sub-optimal measures such as the Levenshtein edit distance to detect cognates, whereby we demonstrate that the longest common sub-sequence is linguistically more sound and improves the performance of bilingual lexicon induction. We show that our approach can increase the accuracy of bilingual lexicon induction methods on these languages many times, making bilingual lexicon induction approaches feasible for such under-resourced languages.
Code (0)
등록된 구현이 없습니다.
Tasks
Bilingual Lexicon InductionWord EmbeddingsSimilar Papers 제목 키워드 기반
Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport
Bilingual lexicons form a critical component of various natural language processing applications, including unsupervised and semisupervised machine translation and crosslingual information retrieval. We improve bilingual…
Bilingual Lexicon InductionGraph MatchingInformation RetrievalMachine Translation+2Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport
Bilingual lexicons form a critical component of various NLP applications, including unsupervised and semisupervised machine translation and crosslingual information retrieval. In this work, we improve bilingual lexicon i…
Bilingual Lexicon InductionGraph MatchingInformation RetrievalMachine Translation+2Evaluating bilingual word embeddings on the long tail
Bilingual word embeddings are useful for bilingual lexicon induction, the task of mining translations of given words. Many studies have shown that bilingual word embeddings perform well for bilingual lexicon induction bu…
Bilingual Lexicon InductionMachine TranslationWord EmbeddingsHow Lexical is Bilingual Lexicon Induction?
In contemporary machine learning approaches to bilingual lexicon induction (BLI), a model learns a mapping between the embedding spaces of a language pair. Recently, retrieve-and-rank approach to BLI has achieved state o…
Bilingual Lexicon InductionImproving Bilingual Lexicon Induction for Low Frequency Words
This paper designs a Monolingual Lexicon Induction task and observes that two factors accompany the degraded accuracy of bilingual lexicon induction for rare words. First, a diminishing margin between similarities in low…
Bilingual Lexicon Induction