SINA-BERT: A Pre-Trained Language Model for Analysis of Medical Texts in Persian
We have released SINA-BERT, a language model pre-trained on BERT to address the lack of a high-quality Persian language model in the medical domain. SINA-BERT utilizes pre-training on a large-scale corpus of medical contents including formal and informal texts collected from various online resources in order to improve the performance on health-care related tasks. We employ SINA-BERT to complete following representative tasks: categorization of medical questions, medical sentiment analysis, medical named entity recognition, and medical question retrieval. For each task, we have developed Persian annotated data sets for training and evaluation and learnt a representation for the data of each task especially complex and long medical questions. With the same architecture being used in each task, SINA-BERT outperforms BERT-based models that were previously made available in the Persian language.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMedical Named Entity Recognitionnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)RetrievalSentiment AnalysisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SINA-BERT: A pre-trained Language Model for Analysis of Medical Texts in Persian
We have released Sina-BERT, a language model pre-trained on BERT (Devlin et al., 2018) to address the lack of a high-quality Persian language model in the medical domain. SINA-BERT utilizes pre-training on a large-scale …
Language ModelingLanguage ModellingRetrievalSentiment AnalysisSINAI@SMM4H’22: Transformers for biomedical social media text mining in Spanish
This paper covers participation of the SINAI team in Tasks 5 and 10 of the Social Media Mining for Health (#SSM4H) workshop at COLING-2022. These tasks focus on leveraging Twitter posts written in Spanish for healthcare …
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical liter…
Drug–drug Interaction ExtractionFew-Shot LearningLanguage ModellingMedical Named Entity Recognition+8Comparative Analysis of Fine-tuned Deep Learning Language Models for ICD-10 Classification Task for Bulgarian Language
The task of automatic diagnosis encoding into standard medical classifications and ontologies, is of great importance in medicine - both to support the daily tasks of physicians in the preparation and reporting of clinic…
Sesame Street to Mount Sinai: BERT-constrained character-level Moses models for multilingual lexical normalization
This paper describes the HEL-LJU submissions to the MultiLexNorm shared task on multilingual lexical normalization. Our system is based on a BERT token classification preprocessing step, where for each token the type of …
Lexical Normalizationtoken-classificationToken Classification