Language Modelling on LAMBADA

37개 결과 · ⬇ CSV · JSON

Accuracy

Rank	Model	Accuracy	Perplexity	Paper	Code	Year
1	PaLM-540B (Few-Shot)	89.7	–	PaLM: Scaling Language Modeling with Pathways	lucidrains/CoCa-pytorch · lucidrains/PaLM-pytorch · google/paxml · +4	2022
2	PaLM 2-L (one-shot)	86.9	–	PaLM 2 Technical Report	eternityyw/tram-benchmark	2023
3	GPT-3 175B (Few-Shot)	86.4	1.92	Language Models are Few-Shot Learners	ggml-org/llama.cpp · ggerganov/llama.cpp · karpathy/llm.c · +64	2020
4	LLaMA-65B+CFG (Zero-Shot)	84.0	–	Stay on topic with Classifier-Free Guidance		2023
5	LLaMA-30B+CFG (zero-shot)	83.9	–	Stay on topic with Classifier-Free Guidance		2023
6	PaLM 2-M (one-shot)	83.7	–	PaLM 2 Technical Report	eternityyw/tram-benchmark	2023
7	Cohere Large	82.33	–
8	LLaMA-13B+CFG (zero-shot)	82.2	–	Stay on topic with Classifier-Free Guidance		2023
9	PaLM-540B (One-Shot)	81.8	–	PaLM: Scaling Language Modeling with Pathways	lucidrains/CoCa-pytorch · lucidrains/PaLM-pytorch · google/paxml · +4	2022
10	GLaM 62B/64E (One-Shot)	80.9	–	GLaM: Efficient Scaling of Language Models with Mixture-of-Experts		2021
11	PaLM 2-S (one-shot)	80.7	–	PaLM 2 Technical Report	eternityyw/tram-benchmark	2023
12	GLM-130B (bidirectional attention)	80.2	–	GLM-130B: An Open Bilingual Pre-trained Model	thudm/chatglm2-6b · thudm/chatglm3 · thudm/chatglm · +6	2022
13	SparseGPT (175B, 2:4 Sparsity)	79.47	–	SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot	nvidia/tensorrt-model-optimizer · ist-daslab/sparsegpt · nvlabs/maskllm · +3	2023
14	SparseGPT (175B, 4:8 Sparsity)	78.77	–	SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot	nvidia/tensorrt-model-optimizer · ist-daslab/sparsegpt · nvlabs/maskllm · +3	2023
15	PaLM-540B (Zero-Shot)	77.9	–	PaLM: Scaling Language Modeling with Pathways	lucidrains/CoCa-pytorch · lucidrains/PaLM-pytorch · google/paxml · +4	2022
16	Chinchilla (Zero-Shot)	77.7	–	Training Compute-Optimal Large Language Models	karpathy/llama2.c · nkluge-correa/teenytinyllama	2022
17	SparseGPT (175B, 50% Sparsity)	76.51	–	SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot	nvidia/tensorrt-model-optimizer · ist-daslab/sparsegpt · nvlabs/maskllm · +3	2023
18	GPT-3 175B (Zero-Shot)	76.2	3.00	Language Models are Few-Shot Learners	ggml-org/llama.cpp · ggerganov/llama.cpp · karpathy/llm.c · +64	2020
19	OPT-175B	75.59	–	SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot	nvidia/tensorrt-model-optimizer · ist-daslab/sparsegpt · nvlabs/maskllm · +3	2023
20	GPT-3 13B (Zero-Shot)	72.5	3.56	Language Models are Few-Shot Learners	ggml-org/llama.cpp · ggerganov/llama.cpp · karpathy/llm.c · +64	2020