Word Sense Disambiguation 벤치마크
Word Sense Disambiguation on RUSSE
Accuracy
- 2020-10-29 — Human Benchmark: Accuracy 0.805
| Rank | Model | Accuracy | Paper | Code | Year |
|---|---|---|---|---|---|
| 1 | Human Benchmark | 0.805 | RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark | RussianNLP/RussianSuperGLUE · RussianNLP/MOROCCO | 2020 |
| 2 | ruT5-large-finetune | 0.735 | |||
| 3 | RuBERT conversational | 0.729 | |||
| 4 | RuBERT plain | 0.726 | |||
| 5 | ruRoberta-large finetune | 0.715 | |||
| 6 | ruBert-base finetune | 0.706 | |||
| 7 | Multilingual Bert | 0.69 | |||
| 8 | ruT5-base-finetune | 0.682 | |||
| 8 | ruBert-large finetune | 0.682 | |||
| 10 | SBERT_Large_mt_ru_finetuning | 0.657 | |||
| 11 | SBERT_Large | 0.654 | |||
| 12 | RuGPT3Large | 0.647 | |||
| 13 | RuGPT3Medium | 0.642 | |||
| 14 | MT5 Large | 0.633 | |||
| 15 | heuristic majority | 0.595 | Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks | 2021 | |
| 16 | Golden Transformer | 0.587 | |||
| 16 | YaLM 1.0B few-shot | 0.587 | |||
| 16 | majority_class | 0.587 | Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks | 2021 | |
| 19 | RuGPT3Small | 0.57 | |||
| 19 | Baseline TF-IDF1.1 | 0.57 | RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark | RussianNLP/RussianSuperGLUE · RussianNLP/MOROCCO | 2020 |