N-gram-like Language Models Predict Naturalistic Reading Time Best

2026-03-10 · James A. Michaelov, Roger P. Levy arxiv

Recent work has found that contemporary language models such as transformers can become so good at next-word prediction that the probabilities they calculate become worse for predicting naturalistic reading time. In this paper, we propose that this can be explained by reading time being shaped by simple n-gram statistics rather than the more complex statistics learned by state-of-the-art transformer language models. We demonstrate that the neural language models whose predictions are most correlated with n-gram probability are also those that calculate probabilities that are the most correlated with eye-tracking-based metrics of reading time on naturalistic text.

📄 PDF Abstract BibTeX arXiv:2603.09872

Code (0)

등록된 구현이 없습니다.

Language Proficiency Assessment from Eye Movements in Naturalistic Passage Reading

2026-08-31 · Shachar Frenkel, Ido Falah, Omer Shubi, Yevgeni Berzak arxiv

Standard language proficiency tests rely on linguistic tasks such as vocabulary, grammar and reading comprehension quizzes. An alternative, cognitively motivated approach, introduced in Berzak et al. (2018), proposed ins…

Reading Comprehension

Measuring the Impact of (Psycho-)Linguistic and Readability Features and Their Spill Over Effects on the Prediction of Eye Movement Patterns

2022-03-15 · ACL 2022 5 · Daniel Wiechmann, Yu Qiao, Elma Kerz, Justus Mattern

There is a growing interest in the combined use of NLP and machine learning methods to predict gaze patterns during naturalistic reading. While promising results have been obtained through the use of transformer-based la…

On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior

2020-06-02 · Ethan Gotlieb Wilcox, Jon Gauthier, Jennifer Hu, Peng Qian 외

Human reading behavior is tuned to the statistics of natural language: the time it takes human subjects to read a word can be predicted from estimates of the word's probability in context. However, it remains an open que…

Open-Ended Question Answering

An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal

2026-04-20 · Ryo Yoshida, Shinnosuke Isono, Taiga Someya, Yohei Oseki 외 arxiv

Surprisal theory hypothesizes that the difficulty of human sentence processing increases linearly with surprisal, the negative log-probability of a word given its context. Computational psycholinguistics has tested this …

Information-Theoretic Storage Cost in Sentence Comprehension

2026-02-20 · Kohei Kajikawa, Shinnosuke Isono, Ethan Gotlieb Wilcox arxiv

Real-time sentence comprehension imposes a significant load on working memory, as comprehenders must maintain contextual information to anticipate future input. While measures of such load have played an important role i…