How to Pre-Train Your Model? Comparison of Different Pre-Training Models for Biomedical Question Answering
Using deep learning models on small scale datasets would result in overfitting. To overcome this problem, the process of pre-training a model and fine-tuning it to the small scale dataset has been used extensively in domains such as image processing. Similarly for question answering, pre-training and fine-tuning can be done in several ways. Commonly reading comprehension models are used for pre-training, but we show that other types of pre-training can work better. We compare two pre-training models based on reading comprehension and open domain question answering models and determine the performance when fine-tuned and tested over BIOASQ question answering dataset. We find open domain question answering model to be a better fit for this task rather than reading comprehension model.
Code (0)
등록된 구현이 없습니다.
Tasks
Open-Domain Question AnsweringQuestion AnsweringReading ComprehensionSimilar Papers 제목 키워드 기반
A Comprehensive Evaluation of Large Language Models on Benchmark Biomedical Text Processing Tasks
Recently, Large Language Models (LLM) have demonstrated impressive capability to solve a wide range of tasks. However, despite their success across various tasks, no prior work has investigated their capability in the bi…
BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity Recognition
Using language models (LMs) pre-trained in a self-supervised setting on large corpora and then fine-tuning for a downstream task has helped to deal with the problem of limited label data for supervised learning tasks suc…
Language ModelingLanguage Modellingnamed-entity-recognitionNamed Entity Recognition+2Method and Software Tool for Generating Artificial Databases of Biomedical Images Based on Deep Neural Networks
A wide variety of biomedical image data, as well as methods for generating training images using basic deep neural networks, were analyzed. Additionally, all platforms for creating images were analyzed, considering their…
Image GenerationIdentifying Comparative Structures in Biomedical Text
Comparison sentences are very commonly used by authors in biomedical literature to report results of experiments. In such comparisons, authors typically make observations under two different scenarios. In this paper, we …
SentenceBuilding Diversified Multiple Trees for Classification in High Dimensional Noisy Biomedical Data
It is common that a trained classification model is applied to the operating data that is deviated from the training data because of noise. This paper demonstrates that an ensemble classifier, Diversified Multiple Tree (…
General Classification