Slot Filling 벤치마크
Slot Filling on KILT: T-REx
KILT-AC
- 2020-09-04 — T5-base: KILT-AC 0.0
- 2020-12-23 — DensePhrases: KILT-AC 27.84
- 2022-07-13 — Re2G: KILT-AC 75.84
| Rank | Model | KILT-AC | R-Prec | Recall@5 | Accuracy | F1 | KILT-F1 | Paper | Code | Year |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Re2G | 75.84 | 80.7 | 89.0 | 87.68 | 89.93 | 77.05 | Re2G: Retrieve, Rerank, Generate | ibm/kgi-slot-filling | 2022 |
| 2 | KGI_1 | 69.14 | 74.36 | 83.14 | 84.36 | 87.24 | 70.58 | |||
| 3 | Wikipedia | 64.64 | 75.64 | 87.57 | 81.34 | 84.46 | 66.64 | |||
| 4 | MetaRAG | 61.88 | 66.36 | 76.24 | 78.66 | 81.71 | 63.09 | |||
| 5 | single ngram | 60.08 | 67.8 | 81.52 | 83.72 | 86.53 | 61.72 | |||
| 6 | KGI_0 (reupload) | 55.54 | 59.7 | 70.38 | 77.9 | 81.31 | 56.79 | |||
| 7 | Coop. DistilBert | 36.68 | 48.08 | 51.86 | 49.04 | 54.62 | 39.57 | |||
| 8 | DensePhrases | 27.84 | 37.62 | 40.07 | 53.9 | 61.74 | 32.34 | Learning Dense Representations of Phrases at Scale | princeton-nlp/SimCSE · jhyuklee/DensePhrases · princeton-nlp/DensePhrases · +1 | 2020 |
| 8 | 10k | 27.84 | 37.62 | 40.07 | 53.9 | 61.74 | 32.34 | |||
| 10 | RAG | 23.12 | 28.68 | 33.04 | 59.2 | 62.96 | 23.94 | |||
| 11 | BART + DPR | 11.12 | 13.26 | 17.04 | 59.16 | 62.76 | 11.41 | |||
| 12 | GENRE | 0.04 | 79.42 | 85.33 | 0.1 | 7.67 | 6.66 | |||
| 13 | TABi | 0.0 | 81.9 | 89.36 | 0.0 | 0.0 | 0.0 | |||
| 13 | chriskuei | 0.0 | 79.98 | 85.75 | 0.0 | 0.0 | 0.0 | |||
| 13 | Multi-task DPR | 0.0 | 69.46 | 83.88 | 0.0 | 0.0 | 0.0 | |||
| 13 | Sphere | 0.0 | 0.0 | 0.0 | 57.02 | 61.46 | 0.0 | |||
| 13 | BART | 0.0 | 0.0 | 0.0 | 45.06 | 49.24 | 0.0 | |||
| 13 | T5-base | 0.0 | 0.0 | 0.0 | 43.56 | 50.61 | 0.0 | KILT: a Benchmark for Knowledge Intensive Language Tasks | facebookresearch/KILT · facebookresearch/editeval · zouharvi/kb-shrink | 2020 |
| 13 | multi-task small | 0.0 | 0.0 | 0.0 | 19.3 | 25.81 | 0.0 | |||
| 13 | JivBest | 0.0 | 0.0 | 0.0 | 0.02 | 2.04 | 0.0 |