Common Sense Reasoning on RuCoS

22개 결과 · ⬇ CSV · JSON

Average F1

Rank	Model	Average F1	EM	Paper	Code	Year
1	Human Benchmark	0.93	0.89	RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark	RussianNLP/RussianSuperGLUE · RussianNLP/MOROCCO	2020
2	Golden Transformer	0.92	0.924
3	YaLM 1.0B few-shot	0.86	0.859
4	ruT5-large-finetune	0.81	0.764
5	ruT5-base-finetune	0.79	0.752
6	ruBert-base finetune	0.74	0.716
7	ruRoberta-large finetune	0.73	0.716
8	ruBert-large finetune	0.68	0.658
9	RuGPT3XL few-shot	0.67	0.665
10	MT5 Large	0.57	0.562	mT5: A massively multilingual pre-trained text-to-text transformer	huggingface/transformers · google-research/multilingual-t5 · google-research/byt5 · +5	2020
11	SBERT_Large	0.36	0.351
12	SBERT_Large_mt_ru_finetuning	0.35	0.347
13	RuBERT plain	0.32	0.314
14	Multilingual Bert	0.29	0.29
15	heuristic majority	0.26	0.257	Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks		2021
15	Baseline TF-IDF1.1	0.26	0.252	RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark	RussianNLP/RussianSuperGLUE · RussianNLP/MOROCCO	2020
17	Random weighted	0.25	0.247	Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks		2021
17	majority_class	0.25	0.247	Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks		2021
19	RuGPT3Medium	0.23	0.224
20	RuBERT conversational	0.22	0.218