Multiple Choice Question Answering (MCQA) 벤치마크
Multiple Choice Question Answering (MCQA) on MedMCQA
Test Set (Acc-%)
- 2022-03-27 — PubmedBERT(Gu et al., 2022): Test Set (Acc-%) 0.41
- 2022-07-17 — Codex 5-shot CoT: Test Set (Acc-%) 0.627
- 2022-09-23 — VOD (BioLinkBERT): Test Set (Acc-%) 0.629
- 2023-05-16 — Med-PaLM 2 (ER): Test Set (Acc-%) 0.723
| Rank | Model | Test Set (Acc-%) | Dev Set (Acc-%) | Paper | Code | Year |
|---|---|---|---|---|---|---|
| 21 | OPT (few-shot, k=5) | – | 0.296 | Galactica: A Large Language Model for Science | paperswithcode/galai | 2022 |
| 22 | PaLM (8B, Few-shot) | – | 0.267 | Large Language Models Encode Clinical Knowledge | dmis-lab/olaph | 2022 |