paper-with-me

Question Answering 벤치마크

Question Answering on MultiRC

30개 결과 · ⬇ CSV · JSON

F1

18.8 36.62 54.45 72.27 90.1 2018-10 2026-09 BERT-large(single model) — 70.0 (2018-10-11) T5-XXL 11B (fine-tuned) — 88.1 (2019-10-23) GPT-3 175B (Few-Shot) — 75.4 (2020-05-28) DeBERTa-1.5B — 88.2 (2020-06-05) FLAN 137B (prompt-tuned) — 83.4 (2021-09-03) FLAN 137B (zero-shot) — 77.5 (2021-09-03) FLAN 137B (1-shot) — 72.1 (2021-09-03) KELM (finetuning BERT-large based single model) — 70.8 (2021-09-09) ST-MoE-32B 269B (fine-tuned) — 89.6 (2022-02-17) ST-MoE-L 4.1B (fine-tuned) — 86.0 (2022-02-17) PaLM 540B (finetuned) — 90.1 (2022-04-05) N-Grammer 343M — 62.0 (2022-07-13) AlexaTM 20B — 59.6 (2022-08-02) Neo-6B (QA + WS) — 63.8 (2022-10-05) Neo-6B (few-shot) — 60.8 (2022-10-05) Neo-6B (QA) — 58.8 (2022-10-05) Turing NLR v5 XXL 5.4B (fine-tuned) — 88.4 (2022-12-04) Vega v2 6B (fine-tuned) — 88.2 (2022-12-04) Bloomberg GPT 50B (1-shot) — 62.3 (2023-03-30) BLOOM 176B (1-shot) — 26.7 (2023-03-30) GPT-NeoX 20B (1-shot) — 22.9 (2023-03-30) OPT 66B (1-shot) — 18.8 (2023-03-30) PaLM 2-L (one-shot) — 88.2 (2023-05-17) PaLM 2-M (one-shot) — 84.1 (2023-05-17) PaLM 2-S (one-shot) — 84.0 (2023-05-17) BERT-large(single model) — 70.0 (2018-10-11) T5-XXL 11B (fine-tuned) — 88.1 (2019-10-23) DeBERTa-1.5B — 88.2 (2020-06-05) ST-MoE-32B 269B (fine-tuned) — 89.6 (2022-02-17) PaLM 540B (finetuned) — 90.1 (2022-04-05)
RankModel F1EM PaperCodeYear
1 PaLM 540B (finetuned) 90.169.2 PaLM: Scaling Language Modeling with Pathways lucidrains/CoCa-pytorch · lucidrains/PaLM-pytorch · google/paxml · +4 2022
2 ST-MoE-32B 269B (fine-tuned) 89.6– ST-MoE: Designing Stable and Transferable Sparse Expert Models tensorflow/mesh · xuefuzhao/openmoe · yikangshen/megablocks 2022
3 Turing NLR v5 XXL 5.4B (fine-tuned) 88.463 Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE 2022
4 DeBERTa-1.5B 88.263.7 DeBERTa: Decoding-enhanced BERT with Disentangled Attention huggingface/transformers · microsoft/DeBERTa · osu-nlp-group/mind2web · +11 2020
4 Vega v2 6B (fine-tuned) 88.262.4 Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE 2022
4 PaLM 2-L (one-shot) 88.2– PaLM 2 Technical Report eternityyw/tram-benchmark 2023
7 T5-XXL 11B (fine-tuned) 88.1– Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer huggingface/transformers · PaddlePaddle/PaddleNLP · google-research/text-to-text-transfer-transformer · +54 2019
8 ST-MoE-L 4.1B (fine-tuned) 86– ST-MoE: Designing Stable and Transferable Sparse Expert Models tensorflow/mesh · xuefuzhao/openmoe · yikangshen/megablocks 2022
9 PaLM 2-M (one-shot) 84.1– PaLM 2 Technical Report eternityyw/tram-benchmark 2023
10 PaLM 2-S (one-shot) 84.0– PaLM 2 Technical Report eternityyw/tram-benchmark 2023
11 FLAN 137B (prompt-tuned) 83.4– Finetuned Language Models Are Zero-Shot Learners hiyouga/llama-efficient-tuning · bigcode-project/starcoder · bigscience-workshop/promptsource · +5 2021
12 FLAN 137B (zero-shot) 77.5– Finetuned Language Models Are Zero-Shot Learners hiyouga/llama-efficient-tuning · bigcode-project/starcoder · bigscience-workshop/promptsource · +5 2021
13 GPT-3 175B (Few-Shot) 75.4– Language Models are Few-Shot Learners ggml-org/llama.cpp · ggerganov/llama.cpp · karpathy/llm.c · +64 2020
14 FLAN 137B (1-shot) 72.1– Finetuned Language Models Are Zero-Shot Learners hiyouga/llama-efficient-tuning · bigcode-project/starcoder · bigscience-workshop/promptsource · +5 2021
15 KELM (finetuning BERT-large based single model) 70.827.2 KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs nlp-anonymous-happy/anonymous-kg-guided-nlp 2021
16 BERT-large(single model) 70.024.1 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding huggingface/transformers · tensorflow/models · labmlai/annotated_deep_learning_paper_implementations · +531 2018
17 Neo-6B (QA + WS) 63.8– Ask Me Anything: A simple strategy for prompting language models hazyresearch/ama_prompting · simran-arora/privacy_fm · simran-arora/focus 2022
18 Bloomberg GPT 50B (1-shot) 62.3– BloombergGPT: A Large Language Model for Finance yangletliu/finlora · open-finance-lab/finlora 2023
19 N-Grammer 343M 6211.3 N-Grammer: Augmenting Transformers with latent n-grams tensorflow/lingvo · yiyixuxu/n-grammer-flax 2022
20 Neo-6B (few-shot) 60.8– Ask Me Anything: A simple strategy for prompting language models hazyresearch/ama_prompting · simran-arora/privacy_fm · simran-arora/focus 2022
21 AlexaTM 20B 59.6– AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model amazon-science/alexa-teacher-models 2022
22 Neo-6B (QA) 58.8– Ask Me Anything: A simple strategy for prompting language models hazyresearch/ama_prompting · simran-arora/privacy_fm · simran-arora/focus 2022
23 BLOOM 176B (1-shot) 26.7– BloombergGPT: A Large Language Model for Finance yangletliu/finlora · open-finance-lab/finlora 2023
24 GPT-NeoX 20B (1-shot) 22.9– BloombergGPT: A Large Language Model for Finance yangletliu/finlora · open-finance-lab/finlora 2023
25 OPT 66B (1-shot) 18.8– BloombergGPT: A Large Language Model for Finance yangletliu/finlora · open-finance-lab/finlora 2023
26 T5-11B –63.3 Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer huggingface/transformers · PaddlePaddle/PaddleNLP · google-research/text-to-text-transfer-transformer · +54 2019
27 Hybrid H3 355M (3-shot, logit scoring) –59.7 Hungry Hungry Hippos: Towards Language Modeling with State Space Models hazyresearch/safari · hazyresearch/h3 · lindermanlab/S5 2022
28 Hybrid H3 355M (0-shot, logit scoring) –59.5 Hungry Hungry Hippos: Towards Language Modeling with State Space Models hazyresearch/safari · hazyresearch/h3 · lindermanlab/S5 2022
29 Hybrid H3 125M (0-shot, logit scoring) –51.4 Hungry Hungry Hippos: Towards Language Modeling with State Space Models hazyresearch/safari · hazyresearch/h3 · lindermanlab/S5 2022
30 Hybrid H3 125M (3-shot, logit scoring) –48.9 Hungry Hungry Hippos: Towards Language Modeling with State Space Models hazyresearch/safari · hazyresearch/h3 · lindermanlab/S5 2022
1–30 / 30 페이지당 10 20 50 100