Context based lemmatizer for Polish language
Lemmatization is the process of grouping together the inflected forms of a word so they can be analysed as a single item, identified by the word's lemma, or dictionary form. In computational linguistics, lemmatisation is the algorithmic process of determining the lemma of a word based on its intended meaning. Unlike stemming, lemmatisation depends on correctly identifying the intended part of speech and meaning of a word in a sentence, as well as within the larger context surrounding that sentence. As a result, developing efficient lemmatisation algorithm is the complex task. In recent years it can be observed that deep learning models used for this task outperform other methods including machine learning algorithms. In this paper the polish lemmatizer based on Google T5 model is presented. The training was run with different context lengths. The model achieves the best results for polish language lemmatisation process.
Code (0)
등록된 구현이 없습니다.
Tasks
LEMMALemmatizationSentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Evaluating Shortest Edit Script Methods for Contextual Lemmatization
Modern contextual lemmatizers often rely on automatically induced Shortest Edit Scripts (SES), namely, the number of edit operations to transform a word form into its lemma. In fact, different methods of computing SES ha…
LEMMALemmatizationtoken-classificationToken ClassificationOn the Role of Morphological Information for Contextual Lemmatization
Lemmatization is a natural language processing (NLP) task which consists of producing, from a given inflected word, its canonical form or lemma. Lemmatization is one of the basic tasks that facilitate downstream NLP appl…
LEMMALemmatizationUniversal Lemmatizer: A Sequence to Sequence Model for Lemmatizing Universal Dependencies Treebanks
In this paper we present a novel lemmatization method based on a sequence-to-sequence neural network architecture and morphosyntactic context representation. In the proposed method, our context-sensitive lemmatizer gener…
Data AugmentationLEMMALemmatizationA Neural Lemmatizer for Bengali
We propose a novel neural lemmatization model which is language independent and supervised in nature. To handle the words in a neural framework, word embedding technique is used to represent words as vectors. The propose…
LEMMALemmatizationTraining Data Augmentation for Context-Sensitive Neural Lemmatizer Using Inflection Tables and Raw Text
Lemmatization aims to reduce the sparse data problem by relating the inflected forms of a word to its dictionary form. Using context can help, both for unseen and ambiguous words. Yet most context-sensitive approaches re…
Data AugmentationLEMMALemmatizationSentence