Auto Debugging 벤치마크
Auto Debugging on Big-bench Lite
Exact string match
- 2022-04-05 — PaLM 62B (few-shot, k=5): Exact string match 38.2
| Rank | Model | Exact string match | Paper | Code | Year |
|---|---|---|---|---|---|
| 1 | PaLM 62B (few-shot, k=5) | 38.2 | PaLM: Scaling Language Modeling with Pathways | lucidrains/CoCa-pytorch · lucidrains/PaLM-pytorch · google/paxml · +4 | 2022 |
| 1 | PaLM 540B (few-shot, k=5) | 38.2 | PaLM: Scaling Language Modeling with Pathways | lucidrains/CoCa-pytorch · lucidrains/PaLM-pytorch · google/paxml · +4 | 2022 |
| 3 | PaLM 8B (few-shot, k=5) | 14.7 | PaLM: Scaling Language Modeling with Pathways | lucidrains/CoCa-pytorch · lucidrains/PaLM-pytorch · google/paxml · +4 | 2022 |