RealMedQA: A pilot biomedical question answering dataset containing realistic clinical questions
Clinical question answering systems have the potential to provide clinicians with relevant and timely answers to their questions. Nonetheless, despite the advances that have been made, adoption of these systems in clinical settings has been slow. One issue is a lack of question-answering datasets which reflect the real-world needs of health professionals. In this work, we present RealMedQA, a dataset of realistic clinical questions generated by humans and an LLM. We describe the process for generating and verifying the QA pairs and assess several QA models on BioASQ and RealMedQA to assess the relative difficulty of matching answers to questions. We show that the LLM is more cost-efficient for generating "ideal" QA pairs. Additionally, we achieve a lower lexical similarity between questions and answers than BioASQ which provides an additional challenge to the top two QA models, as per the results. We release our code and our dataset publicly to encourage further research.
Code (1)
Tasks
Question AnsweringSimilar Papers 제목 키워드 기반
Overview of TREC 2024 Biomedical Generative Retrieval (BioGen) Track
With the advancement of large language models (LLMs), the biomedical domain has seen significant progress and improvement in multiple tasks such as biomedical question answering, lay language summarization of the biomedi…
Medical Question AnsweringQuestion AnsweringRetrievalContextual embedding and model weighting by fusing domain knowledge on Biomedical Question Answering
Biomedical Question Answering aims to obtain an answer to the given question from the biomedical domain. Due to its high requirement of biomedical domain knowledge, it is difficult for the model to learn domain knowledge…
Cloze TestMachine Reading ComprehensionQuestion AnsweringUnsupervised Pre-trainingBiomedical Multi-hop Question Answering Using Knowledge Graph Embeddings and Language Models
Biomedical knowledge graphs (KG) are heterogenous networks consisting of biological entities as nodes and relations between them as edges. These entities and relations are extracted from millions of research papers and u…
Knowledge Graph EmbeddingsKnowledge GraphsMulti-hop Question AnsweringQuestion AnsweringTop K Relevant Passage Retrieval for Biomedical Question Answering
Question answering is a task that answers factoid questions using a large collection of documents. It aims to provide precise answers in response to the user's questions in natural language. Question answering relies on …
ArticlesPassage RetrievalQuestion AnsweringReading Comprehension+1Pre-trained Language Model for Biomedical Question Answering
The recent success of question answering systems is largely attributed to pre-trained language models. However, as language models are mostly pre-trained on general domain corpora such as Wikipedia, they often have diffi…
Language ModelingLanguage ModellingmodelQuestion Answering