DadmaTools: Natural Language Processing Toolkit for Persian Language
We introduce DadmaTools, an open-source Python Natural Language Processing toolkit for the Persian language. The toolkit is a neural pipeline based on spaCy for several text processing tasks, including normalization, tokenization, lemmatization, part-of-speech, dependency parsing, constituency parsing, chunking, and ezafe detecting. DadmaTools relies on fine-tuning of ParsBERT using the PerDT dataset for most of the tasks. Dataset module and embedding module are included in DadmaTools that support different Persian datasets, embeddings, and commonly used functions for them. Our evaluations show that DadmaTools can attain state-of-the-art performance on multiple NLP tasks. The source code is freely available at https://github.com/Dadmatech/DadmaTools.
Code (1)
Tasks
ChunkingConstituency ParsingDependency ParsingLemmatizationSimilar Papers 제목 키워드 기반
ParsiNorm: A Persian Toolkit for Speech Processing Normalization
In general, speech processing models consist of a language model along with an acoustic model. Regardless of the language model's complexity and variants, three critical pre-processing steps are needed in language models…
Language ModellingSentenceParsivar: A Language Processing Toolkit for Persian
ParsiPy: NLP Toolkit for Historical Persian Texts in Python
The study of historical languages presents unique challenges due to their complex orthographic systems, fragmentary textual evidence, and the absence of standardized digital representations of text in those languages. Ta…
LemmatizationPart-Of-Speech TaggingTransliterationPersianLLaMA: Towards Building First Persian Large Language Model
Despite the widespread use of the Persian language by millions globally, limited efforts have been made in natural language processing for this language. The use of large language models as effective tools in various nat…
Language ModelingLanguage ModellingLarge Language ModelMachine Translation+5A Knowledge-based Approach for Answering Complex Questions in Persian
Research on open-domain question answering (QA) has a long tradition. A challenge in this domain is answering complex questions (CQA) that require complex inference methods and large amounts of knowledge. In low resource…
Open-Domain Question AnsweringQuestion Answering