Subword-based Cross-lingual Transfer of Embeddings from Hindi to Marathi

2021-10-16 · ACL ARR October 2021 10 · Anonymous

Word embeddings are growing to be a crucial resource in the field of NLP for any language. This work focuses on static subword embeddings transfer for Indian languages from a relatively higher resource language to a genealogically related low resource language. We work with Hindi-Marathi as our language pair, simulating a low-resource scenario for Marathi. We demonstrate the consistent benefits of unsupervised morphemic segmentation on both source and target sides over the treatment performed by FastText. We show that a trivial "copy-and-paste'' embeddings transfer based on even perfect bilingual lexicons is inadequate in capturing language-specific relationships. Our best-performing approach uses an EM-style approach to learning bilingual subword embeddings; the resulting embeddings are evaluated using the publicly available Marathi Word Similarity task as well as WordNet-Based Synonymy Tests. We find that our approach significantly outperforms the FastText baseline on both tasks; on the former task, its performance is close to that of pretrained FastText Marathi embeddings that use two orders of magnitude more Marathi data.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Cross-Lingual TransferWord EmbeddingsWord Similarity

Methods 이 논문이 사용한 방법론

fastText fastText embeddings exploit subword information to construct word embeddings. Representations are learnt of character $n$-grams, and words represented as the sum of the…

Subword-based Cross-lingual Transfer of Embeddings from Hindi to Marathi and Nepali

2022-07-01 · NAACL (SIGMORPHON) 2022 7 · Niyata Bafna, Zdeněk Žabokrtský

Word embeddings are growing to be a crucial resource in the field of NLP for any language. This work introduces a novel technique for static subword embeddings transfer for Indic languages from a relatively higher resour…

Cross-Lingual Transfer Word Embeddings Word Similarity

Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations

2018-08-28 · EMNLP 2018 10 · Aditi Chaudhary, Chunting Zhou, Lori Levin, Graham Neubig 외

Much work in Natural Language Processing (NLP) has been for resource-rich languages, making generalization to new, less-resourced languages challenging. We present two approaches for improving generalization to low-resou…

Avg Machine Translation named-entity-recognition Named Entity Recognition+5

Tomato, Tomahto, Tomate: Measuring the Role of Shared Semantics among Subwords in Multilingual Language Models

2024-11-07 · Xinyu Zhang, Jing Lu, Vinh Q. Tran, Tal Schuster 외

Human understanding of language is robust to different word choices as far as they represent similar semantic concepts. To what extent does our human intuition transfer to language models, which represent all subwords as…

Subword Mapping and Anchoring across Languages

2021-09-09 · Findings (EMNLP) 2021 11 · Giorgos Vernikos, Andrei Popescu-Belis

State-of-the-art multilingual systems rely on shared vocabularies that sufficiently cover all considered languages. To this end, a simple and frequently used approach makes use of subword vocabularies constructed jointly…

Cross-Lingual Natural Language Inference Machine Translation Natural Language Inference Translation

WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models

2021-12-13 · NAACL 2022 7 · Benjamin Minixhofer, Fabian Paischer, Navid Rekabsaz

Large pretrained language models (LMs) have become the central building block of many NLP applications. Training these models requires ever more computational resources and most of the existing models are trained on Engl…

Cross-Lingual Transfer Word Embeddings