Multiple Choice Question Answering (MCQA) 벤치마크

Multiple Choice Question Answering (MCQA) on MedMCQA

22개 결과 · ⬇ CSV · JSON

Test Set (Acc-%)

Rank	Model	Test Set (Acc-%)	Dev Set (Acc-%)	Paper	Code	Year
1	Med-PaLM 2 (ER)	0.723	–	Towards Expert-Level Medical Question Answering with Large Language Models	m42-health/med42	2023
2	Med-PaLM 2 (CoT+SC)	0.715	–	Towards Expert-Level Medical Question Answering with Large Language Models	m42-health/med42	2023
3	Med-PaLM 2 (5-shot)	0.713	–	Towards Expert-Level Medical Question Answering with Large Language Models	m42-health/med42	2023
4	VOD (BioLinkBERT)	0.629	0.583	Variational Open-Domain Question Answering	VodLM/vod · findzebra/fz-openqa	2022
5	Codex 5-shot CoT	0.627	0.597	Can large language models reason about medical questions?	vlievin/medical-reasoning	2022
6	BioMedGPT-10B	0.514	–	BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine	pharmolix/openbiomed	2023
7	PubmedBERT(Gu et al., 2022)	0.41	0.40	MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering	MedMCQA/MedMCQA	2022
8	SciBERT (Beltagy et al., 2019)	0.39	0.39	MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering	MedMCQA/MedMCQA	2022
9	BioBERT (Lee et al.,2020)	0.37	0.38	MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering	MedMCQA/MedMCQA	2022
10	BERT (Devlin et al., 2019)-Base	0.33	0.35	MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering	MedMCQA/MedMCQA	2022
11	Meditron-70B (CoT + SC)	–	66.0	MEDITRON-70B: Scaling Medical Pretraining for Large Language Models	epfllm/meditron	2023
12	Flan-PaLM (540B, SC)	–	0.576	Large Language Models Encode Clinical Knowledge	dmis-lab/olaph	2022
13	Flan-PaLM (540B, Few-shot)	–	0.565	Large Language Models Encode Clinical Knowledge	dmis-lab/olaph	2022
14	PaLM (540B, Few-shot)	–	0.545	Large Language Models Encode Clinical Knowledge	dmis-lab/olaph	2022
15	Flan-PaLM (540B, CoT)	–	0.536	Large Language Models Encode Clinical Knowledge	dmis-lab/olaph	2022
16	GAL 120B (zero-shot)	–	0.529	Galactica: A Large Language Model for Science	paperswithcode/galai	2022
17	Flan-PaLM (62B, Few-shot)	–	0.462	Large Language Models Encode Clinical Knowledge	dmis-lab/olaph	2022
18	PaLM (62B, Few-shot)	–	0.434	Large Language Models Encode Clinical Knowledge	dmis-lab/olaph	2022
19	Flan-PaLM (8B, Few-shot)	–	0.345	Large Language Models Encode Clinical Knowledge	dmis-lab/olaph	2022
20	BLOOM (few-shot, k=5)	–	0.325	Galactica: A Large Language Model for Science	paperswithcode/galai	2022