Surprisal Estimators for Human Reading Times Need Character Models
While the use of character models has been popular in NLP applications, it has not been explored much in the context of psycholinguistic modeling. This paper presents a character model that can be applied to a structural parser-based processing model to calculate word generation probabilities. Experimental results show that surprisal estimates from a structural processing model using this character model deliver substantially better fits to self-paced reading, eye-tracking, and fMRI data than those from large-scale language models trained on much more data. This may suggest that the proposed processing model provides a more humanlike account of sentence processing, which assumes a larger role of morphology, phonotactics, and orthographic complexity than was previously thought.
Code (1)
Tasks
SentenceSimilar Papers 제목 키워드 기반
Why Does Surprisal From Smaller GPT-2 Models Provide Better Fit to Human Reading Times?
This work presents an in-depth analysis of an observation that contradicts the findings of recent work in computational psycholinguistics, namely that smaller GPT-2 models that show higher test perplexity nonetheless gen…
regressionAn Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal
Surprisal theory hypothesizes that the difficulty of human sentence processing increases linearly with surprisal, the negative log-probability of a word given its context. Computational psycholinguistics has tested this …
Temperature-scaling surprisal estimates improve fit to human reading times -- but does it do so for the "right reasons"?
A wide body of evidence shows that human language processing difficulty is predicted by the information-theoretic measure surprisal, a word's negative log probability in context. However, it is still unclear how to best …
Language ModellingLarge Language ModelThe Effect of Surprisal on Reading Times in Information Seeking and Repeated Reading
The effect of surprisal on processing difficulty has been a central topic of investigation in psycholinguistics. Here, we use eyetracking data to examine three language processing regimes that are common in daily life bu…
Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokens
Recent psycholinguistic studies have drawn conflicting conclusions about the relationship between the quality of a language model and the ability of its surprisal estimates to predict human reading times, which has been …
Language ModelingLanguage Modelling