Stem-driven Language Models for Morphologically Rich Languages
Neural language models (LMs) have shown to benefit significantly from enhancing word vectors with subword-level information, especially for morphologically rich languages. This has been mainly tackled by providing subword-level information as an input; using subword units in the output layer has been far less explored. In this work, we propose LMs that are cognizant of the underlying stems in each word. We derive stems for words using a simple unsupervised technique for stem identification. We experiment with different architectures involving multi-task learning and mixture models over words and stems. We focus on four morphologically complex languages -- Hindi, Tamil, Kannada and Finnish -- and observe significant perplexity gains with using our stem-driven LMs when compared with other competitive baseline models.
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-Task LearningSimilar Papers 제목 키워드 기반
Data-Driven Morphological Analysis and Disambiguation for Morphologically Rich Languages and Universal Dependencies
Parsing texts into universal dependencies (UD) in realistic scenarios requires infrastructure for the morphological analysis and disambiguation (MA{\&}D) of typologically different languages as a first tier. MA{\&}D is p…
Morphological AnalysisTAGExplicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew
Pre-trained language models (PLMs) have shown remarkable successes in acquiring a wide range of linguistic knowledge, relying solely on self-supervised training on text streams. Nevertheless, the effectiveness of this la…
Word Semantic Similarity for Morphologically Rich Languages
In this work, we investigate the role of morphology on the performance of semantic similarity for morphologically rich languages, such as German and Greek. The challenge in processing languages with richer morphology tha…
Semantic SimilaritySemantic Textual SimilarityCross-Lingual Word Embeddings for Morphologically Rich Languages
Cross-lingual word embedding models learn a shared vector space for two or more languages so that words with similar meaning are represented by similar vectors regardless of their language. Although the existing models a…
Cross-Lingual Word EmbeddingsTranslationWord EmbeddingsWord TranslationOne Model is All You Need: ByT5-Sanskrit, a Unified Model for Sanskrit NLP Tasks
Morphologically rich languages are notoriously challenging to process for downstream NLP applications. This paper presents a new pretrained language model, ByT5-Sanskrit, designed for NLP applications involving the morph…
AllDependency ParsingInformation RetrievalLanguage Modelling+4