Memorization 벤치마크
Memorization on BIG-bench (Hindu Knowledge)
Accuracy
- 2021-12-08 — Gopher-280B (few-shot, k=5): Accuracy 80.0
- 2022-04-05 — PaLM-540B (few-shot, k=5): Accuracy 95.4
| Rank | Model | Accuracy | Paper | Code | Year |
|---|---|---|---|---|---|
| 1 | PaLM-540B (few-shot, k=5) | 95.4 | PaLM: Scaling Language Modeling with Pathways | lucidrains/CoCa-pytorch · lucidrains/PaLM-pytorch · google/paxml · +4 | 2022 |
| 2 | Gopher-280B (few-shot, k=5) | 80 | Scaling Language Models: Methods, Analysis & Insights from Training Gopher | allenai/dolma · rvlopes/gloria · bramiozo/PubScience | 2021 |
| 3 | PaLM-62B (few-shot, k=5) | 77.7 | PaLM: Scaling Language Modeling with Pathways | lucidrains/CoCa-pytorch · lucidrains/PaLM-pytorch · google/paxml · +4 | 2022 |