Language Modelling on OpenWebText

12개 결과 · ⬇ CSV · JSON

eval_perplexity 낮을수록 좋음

Rank	Model	eval_perplexity	eval_loss	parameters	Paper	Code	Year
1	MDLM-Prime	15.36	–	131M	Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking		2025
2	ARM	17.54	–	131M	Simple and Effective Masked Diffusion Language Models	kuleshov-group/mdlm · masa-ue/svdd	2024
3	EDLM-coAR	17.58	–	131M	Energy-Based Diffusion Language Models for Text Generation		2024
4	GPT2-Hermite	18.39	2.91	124M	Polynomial, trigonometric, and tropical activations	K-H-Ismail/torchortho	2025
5	GPT2-Tropical	18.64	2.92	124M	Polynomial, trigonometric, and tropical activations	K-H-Ismail/torchortho	2025
6	GPT2-Fourier	18.72	2.93	124M	Polynomial, trigonometric, and tropical activations	K-H-Ismail/torchortho	2025
7	GPT2-GELU	19.24	2.95	124M	Polynomial, trigonometric, and tropical activations	K-H-Ismail/torchortho	2025
8	BD3-LMs	20.73	–	131M	Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models	kuleshov-group/bd3lms · MindSpore-scientific/code-12	2025
9	EDLM-NCE	21.52	–	131M	Energy-Based Diffusion Language Models for Text Generation		2024
10	GenMD4	21.80	–	131M	Simplified and Generalized Masked Diffusion for Discrete Data	google-deepmind/md4	2024
11	MDLM	22.98	–	131M	Simple and Effective Masked Diffusion Language Models	kuleshov-group/mdlm · masa-ue/svdd	2024
12	SEDD	24.10	–	131M	Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution	louaaron/score-entropy-discrete-diffusion · ozekri/SEPO · lingxiaoshawn/usd3 · +1	2023