paper-with-me

Question Answering 벤치마크

Question Answering on SQuAD1.1 dev

55개 결과 · ⬇ CSV · JSON

EM

59.95 67.48 75 82.53 90.06 2016-08 2026-09 Match-LSTM with Bi-Ans-Ptr (Boundary+Search+b) — 64.1 (2016-08-29) DCR — 62.5 (2016-10-31) RASOR — 66.4 (2016-11-04) BIDAF (single) — 67.7 (2016-11-05) DCN — 65.4 (2016-11-05) FG fine-grained gate — 59.95 (2016-11-06) MPCM — 66.1 (2016-12-13) SEDT-LSTM — 67.89 (2017-03-02) SECT-LSTM — 67.65 (2017-03-02) FastQAExt (beam-size 5) — 70.3 (2017-03-14) jNet (TreeLSTM adaptation, QTLa, K=100) — 69.1 (2017-03-14) DrQA (Document Reader only) — 69.5 (2017-03-31) Ruminating Reader — 70.6 (2017-04-24) R.M-Reader (single) — 78.9 (2017-05-08) OTF dict+spelling (single) — 63.06 (2017-06-01) R-NET (single) — 71.1 (2017-07-01) DCN (Char + CoVe) — 71.3 (2017-08-01) SRU — 71.4 (2017-09-08) Smarnet — 71.362 (2017-10-08) PhaseCond (single) — 72.1 (2017-10-28) DCN+ (single) — 74.5 (2017-10-31) FusionNet — 75.3 (2017-11-16) SAN (single) — 76.235 (2017-12-10) QANet (data aug x3) — 75.1 (2018-04-23) QANet (data aug x2) — 74.5 (2018-04-23) QANet — 73.6 (2018-04-23) KAR — 76.7 (2018-09-10) BERT-LARGE (Ensemble+TriviaQA) — 86.2 (2018-10-11) BERT-LARGE (Single+TriviaQA) — 84.2 (2018-10-11) FABIR — 65.1 (2018-10-22) XLNet (single model) — 89.7 (2019-06-19) TinyBERT-6 67M — 79.7 (2019-09-23) DistilBERT — 77.7 (2019-10-02) T5-11B — 90.06 (2019-10-23) T5-3B — 88.53 (2019-10-23) T5-Large 770M — 86.66 (2019-10-23) T5-Base — 85.44 (2019-10-23) T5-Small — 79.1 (2019-10-23) XLNet+DSC — 89.79 (2019-11-07) LUKE — 89.8 (2020-10-02) DensePhrases — 78.3 (2020-12-23) BERT-Large-uncased-PruneOFA (90% unstruct sparse) — 83.35 (2021-11-10) BERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8) — 83.22 (2021-11-10) BERT-Base-uncased-PruneOFA (85% unstruct sparse) — 81.1 (2021-11-10) BERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8) — 80.84 (2021-11-10) BERT-Base-uncased-PruneOFA (90% unstruct sparse) — 79.83 (2021-11-10) DistilBERT-uncased-PruneOFA (85% unstruct sparse) — 78.1 (2021-11-10) DistilBERT-uncased-PruneOFA (85% unstruct sparse, QAT Int8) — 77.03 (2021-11-10) DistilBERT-uncased-PruneOFA (90% unstruct sparse) — 76.91 (2021-11-10) DistilBERT-uncased-PruneOFA (90% unstruct sparse, QAT Int8) — 75.62 (2021-11-10) Match-LSTM with Bi-Ans-Ptr (Boundary+Search+b) — 64.1 (2016-08-29) RASOR — 66.4 (2016-11-04) BIDAF (single) — 67.7 (2016-11-05) SEDT-LSTM — 67.89 (2017-03-02) FastQAExt (beam-size 5) — 70.3 (2017-03-14) Ruminating Reader — 70.6 (2017-04-24) R.M-Reader (single) — 78.9 (2017-05-08) BERT-LARGE (Ensemble+TriviaQA) — 86.2 (2018-10-11) XLNet (single model) — 89.7 (2019-06-19) T5-11B — 90.06 (2019-10-23)
RankModel EMF1 Extra Training Data PaperCodeYear
1 T5-11B 90.0695.64 ✓ Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer huggingface/transformers · PaddlePaddle/PaddleNLP · google-research/text-to-text-transfer-transformer · +54 2019
2 LUKE 89.8– ✓ LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention huggingface/transformers · PaddlePaddle/PaddleNLP · studio-ousia/luke · +6 2020
3 XLNet+DSC 89.7995.77 ✓ Dice Loss for Data-imbalanced NLP Tasks ShannonAI/dice_loss_for_NLP · fursovia/self-adj-dice · MindCode-4/code-6 · +1 2019
4 XLNet (single model) 89.795.1 ✓ XLNet: Generalized Autoregressive Pretraining for Language Understanding huggingface/transformers · PaddlePaddle/PaddleNLP · zihangdai/xlnet · +24 2019
5 T5-3B 88.5394.95 ✓ Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer huggingface/transformers · PaddlePaddle/PaddleNLP · google-research/text-to-text-transfer-transformer · +54 2019
6 T5-Large 770M 86.6693.79 Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer huggingface/transformers · PaddlePaddle/PaddleNLP · google-research/text-to-text-transfer-transformer · +54 2019
7 BERT-LARGE (Ensemble+TriviaQA) 86.292.2 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding huggingface/transformers · tensorflow/models · labmlai/annotated_deep_learning_paper_implementations · +531 2018
8 T5-Base 85.4492.08 ✓ Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer huggingface/transformers · PaddlePaddle/PaddleNLP · google-research/text-to-text-transfer-transformer · +54 2019
9 BERT-LARGE (Single+TriviaQA) 84.291.1 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding huggingface/transformers · tensorflow/models · labmlai/annotated_deep_learning_paper_implementations · +531 2018
10 BERT-Large-uncased-PruneOFA (90% unstruct sparse) 83.3590.2 Prune Once for All: Sparse Pre-Trained Language Models intel/intel-extension-for-transformers · intellabs/model-compression-research-package 2021
11 BERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8) 83.2290.02 Prune Once for All: Sparse Pre-Trained Language Models intel/intel-extension-for-transformers · intellabs/model-compression-research-package 2021
12 BERT-Base-uncased-PruneOFA (85% unstruct sparse) 81.188.42 Prune Once for All: Sparse Pre-Trained Language Models intel/intel-extension-for-transformers · intellabs/model-compression-research-package 2021
13 BERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8) 80.8488.24 Prune Once for All: Sparse Pre-Trained Language Models intel/intel-extension-for-transformers · intellabs/model-compression-research-package 2021
14 BERT-Base-uncased-PruneOFA (90% unstruct sparse) 79.8387.25 Prune Once for All: Sparse Pre-Trained Language Models intel/intel-extension-for-transformers · intellabs/model-compression-research-package 2021
15 TinyBERT-6 67M 79.787.5 TinyBERT: Distilling BERT for Natural Language Understanding PaddlePaddle/PaddleNLP · huawei-noah/Pretrained-Language-Model · mindspore-ai/models · +7 2019
16 T5-Small 79.187.24 ✓ Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer huggingface/transformers · PaddlePaddle/PaddleNLP · google-research/text-to-text-transfer-transformer · +54 2019
17 R.M-Reader (single) 78.9 86.3 Reinforced Mnemonic Reader for Machine Reading Comprehension HKUST-KnowComp/MnemonicReader · ewrfcas/Reinforced-Mnemonic-Reader · yly-revive/chainer-mreader 2017
18 DensePhrases 78.386.3 Learning Dense Representations of Phrases at Scale princeton-nlp/SimCSE · jhyuklee/DensePhrases · princeton-nlp/DensePhrases · +1 2020
19 DistilBERT-uncased-PruneOFA (85% unstruct sparse) 78.185.82 Prune Once for All: Sparse Pre-Trained Language Models intel/intel-extension-for-transformers · intellabs/model-compression-research-package 2021
20 DistilBERT 77.7– DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter huggingface/transformers · PaddlePaddle/PaddleNLP · huggingface/swift-coreml-transformers · +34 2019
1–20 / 55 다음 → 페이지당 10 20 50 100