FaBERT: Pre-training BERT on Persian Blogs
We introduce FaBERT, a Persian BERT-base model pre-trained on the HmBlogs corpus, encompassing both informal and formal Persian texts. FaBERT is designed to excel in traditional Natural Language Understanding (NLU) tasks, addressing the intricacies of diverse sentence structures and linguistic styles prevalent in the Persian language. In our comprehensive evaluation of FaBERT on 12 datasets in various downstream tasks, encompassing Sentiment Analysis (SA), Named Entity Recognition (NER), Natural Language Inference (NLI), Question Answering (QA), and Question Paraphrasing (QP), it consistently demonstrated improved performance, all achieved within a compact model size. The findings highlight the importance of utilizing diverse and cleaned corpora, such as HmBlogs, to enhance the performance of language models like BERT in Persian Natural Language Processing (NLP) applications. FaBERT is openly accessible at https://huggingface.co/sbunlp/fabert
Code (0)
등록된 구현이 없습니다.
Tasks
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Natural Language InferenceNatural Language UnderstandingNERQuestion AnsweringSentenceSentiment AnalysisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PerSoMed: A Large-Scale Balanced Dataset for Persian Social Media Text Classification
This research introduces the first large-scale, well-balanced Persian social media text classification dataset, specifically designed to address the lack of comprehensive resources in this domain. The dataset comprises 3…
Text ClassificationData AugmentationHmBlogs: A big general Persian corpus
This paper introduces the hmBlogs corpus for Persian, as a low resource language. This corpus has been prepared based on a collection of nearly 20 million blog posts over a period of about 15 years from a space of Persia…
Matina: A Large-Scale 73B Token Persian Text Corpus
Text corpora are essential for training models used in tasks like summarization, translation, and large language models (LLMs). While various efforts have been made to collect monolingual and multilingual datasets in man…
ArticlesDiversitySINA-BERT: A pre-trained Language Model for Analysis of Medical Texts in Persian
We have released Sina-BERT, a language model pre-trained on BERT (Devlin et al., 2018) to address the lack of a high-quality Persian language model in the medical domain. SINA-BERT utilizes pre-training on a large-scale …
Language ModelingLanguage ModellingRetrievalSentiment AnalysisSINA-BERT: A Pre-Trained Language Model for Analysis of Medical Texts in Persian
We have released SINA-BERT, a language model pre-trained on BERT to address the lack of a high-quality Persian language model in the medical domain. SINA-BERT utilizes pre-training on a large-scale corpus of medical cont…
Language ModelingLanguage ModellingMedical Named Entity Recognitionnamed-entity-recognition+4